Equation 9 · Shrink It, Train It Small, or Search for It: The Main Strategies for Small Models, Compared
What does this equation mean?
Read the formula alongside the article passage below. Each part has a deeper page with its role in the equation, the supporting passage and nearby citations.
the what. Read the equation part by part below; each part has a contextual explanation and a link to its mathematical background.
Read it piece by piece
How to interpret it
Read this expression with the definitions, units, and assumptions supplied by the article.
What the article says around this equation
with N parameters and D training tokens. Chinchilla’s question was: given a fixed compute budget C , what N and D minimize L ? The data-efficient small-model strategy asks a different question with the same law: given a fixed, small N set by the deployment target, what choice of D — and, crucially, what quality of D — minimizes L ? Fixing the small side of the equation first and spending the freed budget on data rather than on parameters is the strategy’s entire premise.
Sources cited in the article section
- [4] Training Compute-Optimal Large Language Models ↗
- [5] Textbooks Are All You Need ↗
- [6] TinyStories: How Small Can Language Models Be and Still Speak Coherent English? ↗
These citations give research context. Read each source to check which claims it supports.