Tagged model compression · show all articles
A capable small model can be reached by compressing a large one, training a small one on purpose, searching architecture space for it, or routing around density altogether. These are four different engineering problems, not four contestants in one race.
A quantized or pruned model reported to lose less than a point of accuracy has been shown to average well across a benchmark. Whether it kept the one capability a deployment actually needs is a separate question the average is built to hide.
Distillation, pruning and quantization are not three names for the same haircut. Each removes a different kind of thing from a trained checkpoint, in a specific order, and each recovers the resulting loss by a different mechanism.