Tagged distillation · show all articles
A starter culture is a population propagated from itself. It runs indefinitely if it is fed and managed, and it drifts or dies if it is only ever fed its own unmanaged output. Synthetic training data behaves the same way.
A datacentre gives a model whatever it asks for. A device gives it a fixed envelope — memory, bandwidth, a sustained thermal budget, a battery — and a model has to be built to live inside it, not merely shrunk until it fits.
Three ways of getting more capability into a servable model, each with a different cost profile, a different failure mode, and a different answer to the question of what a parameter is for.