Filtered to Edge AI & Electronics · show all articles
A model that passes every benchmark on the bench can still fail in the field — not from one bug, but from ten separate ways a device diverges from the conditions the benchmark assumed.
Two questions decide how small and on-device AI matures by 2035: whether dedicated hardware closes the capability gap with the cloud, and whether on-device personalization makes privacy the default. Four scenarios, each with a falsifier.
A capable small model can be reached by compressing a large one, training a small one on purpose, searching architecture space for it, or routing around density altogether. These are four different engineering problems, not four contestants in one race.
A model that answers correctly on the phone sitting on an engineer's desk has cleared the lowest bar in the project. Compression, cloud fallback, fleet testing, versioning and privacy claims are each a decision — made well on purpose, or made badly by default.
A quantized or pruned model reported to lose less than a point of accuracy has been shown to average well across a benchmark. Whether it kept the one capability a deployment actually needs is a separate question the average is built to hide.
Distillation, pruning and quantization are not three names for the same haircut. Each removes a different kind of thing from a trained checkpoint, in a specific order, and each recovers the resulting loss by a different mechanism.
Before a phone could run a language model, it had to run anything at all: a narrower convolution, a dedicated chip, an always-listening detector, and a trick for compressing what a bigger model already knew. This is that decade, in order.