Tagged foundation models · show all articles
Three ways of getting more capability into a servable model, each with a different cost profile, a different failure mode, and a different answer to the question of what a parameter is for.
Decoding is not limited by arithmetic. It is limited by how fast memory can be read, and almost every serving technique in production exists to work around that single fact.
Six problems that more scale has not solved, separated carefully from the many that are merely hard engineering, and the observations that would tell us any of them had been cracked.
A benchmark result is an assay: it depends on how the sample was taken, what impurity was already in the reference, and how many times the balance was read.
A frontier model is a budget divided three ways. Where the compute goes—into weights, into behaviour, or into each individual answer—determines almost everything else about the system.