Tagged foundation models · show all articles
A printed picture exists only when separately produced channels register with one another. Multimodal models are assembled the same way, and the join — not the parts — is where the claim of a single unified system is won or lost.
Two axes govern the next decade of frontier model systems: where the next increment of capability is bought, and whether the physical constraints tighten. Crossing them gives four futures, none of which is the favourite.
The line from Shannon's letter counts to trained deliberation is usually told as one long scaling curve. It was not. It was a sequence of specific bottlenecks, each named, each broken by a different idea.
Serving a frontier model is a thermal and logistical problem before it is a computational one. The binding constraints are bytes moved, joules dissipated, and megawatts delivered — in that order.
A practical guide to the decisions that determine whether an application on a frontier model stays correct and affordable as the model underneath it changes without warning.
The dangerous failures are not the ones that throw an exception. They are the ones where the system keeps returning well-formed output that is quietly wrong, and nothing upstream notices.
Three ways of getting more capability into a servable model, each with a different cost profile, a different failure mode, and a different answer to the question of what a parameter is for.
Decoding is not limited by arithmetic. It is limited by how fast memory can be read, and almost every serving technique in production exists to work around that single fact.
Six problems that more scale has not solved, separated carefully from the many that are merely hard engineering, and the observations that would tell us any of them had been cracked.
A benchmark result is an assay: it depends on how the sample was taken, what impurity was already in the reference, and how many times the balance was read.