Tagged quantization · show all articles
A model that passes every benchmark on the bench can still fail in the field — not from one bug, but from ten separate ways a device diverges from the conditions the benchmark assumed.
A capable small model can be reached by compressing a large one, training a small one on purpose, searching architecture space for it, or routing around density altogether. These are four different engineering problems, not four contestants in one race.
Distillation, pruning and quantization are not three names for the same haircut. Each removes a different kind of thing from a trained checkpoint, in a specific order, and each recovers the resulting loss by a different mechanism.