Tagged on-device AI · show all articles
A model that passes every benchmark on the bench can still fail in the field — not from one bug, but from ten separate ways a device diverges from the conditions the benchmark assumed.
Two questions decide how small and on-device AI matures by 2035: whether dedicated hardware closes the capability gap with the cloud, and whether on-device personalization makes privacy the default. Four scenarios, each with a falsifier.
A model that answers correctly on the phone sitting on an engineer's desk has cleared the lowest bar in the project. Compression, cloud fallback, fleet testing, versioning and privacy claims are each a decision — made well on purpose, or made badly by default.
Distillation, pruning and quantization are not three names for the same haircut. Each removes a different kind of thing from a trained checkpoint, in a specific order, and each recovers the resulting loss by a different mechanism.
Before a phone could run a language model, it had to run anything at all: a narrower convolution, a dedicated chip, an always-listening detector, and a trick for compressing what a bigger model already knew. This is that decade, in order.
A datacentre gives a model whatever it asks for. A device gives it a fixed envelope — memory, bandwidth, a sustained thermal budget, a battery — and a model has to be built to live inside it, not merely shrunk until it fits.