← All parts of this equation

Equation 33 · Part 14 · Shrink It, Train It Small, or Search for It: The Main Strategies for Small Models, Compared

subscript

Clifetime=Creuse⏟compress  +  Ccurate+Ctrain⏟train small  +  Csearch⏟NAS  +  Q⋅cˉtok,withMresident≤BmemC_{\text{lifetime}} = \underbrace{C_{\text{reuse}}}_{\text{compress}} \;+\; \underbrace{C_{\text{curate}} + C_{\text{train}}}_{\text{train small}} \;+\; \underbrace{C_{\text{search}}}_{\text{NAS}} \;+\; Q \cdot \bar c_{\text{tok}}, \qquad \text{with} \quad M_{\text{resident}} \le B_{\text{mem}}
subscript

What this part means

The lower label selects a particular version, component, or indexed member of the quantity. For example, x₀ and xₜ can be values at different positions.

Its job in the formula

A subscript distinguishes a version, component, step, or member of a quantity. It does not automatically mean multiplication.

The passage around this formula

Set side by side, the four strategies do not compete on a single scale, and a comparison that reduces them to one leaderboard number is not describing what any of them actually trades off. Each holds a different quantity fixed as “already spent” and treats a different quantity as the one still to be paid: Clifetime=Creuse⏟compress  +  Ccurate+Ctrain⏟train small  +  Csearch⏟NAS  +  Q⋅cˉtok,withMresident≤BmemC_{\text{lifetime}} = \underbrace{C_{\text{reuse}}}_{\text{compress}} \;+\; \underbrace{C_{\text{curate}} + C_{\text{train}}}_{\text{train small}} \;+\; \underbrace{C_{\text{search}}}_{\text{NAS}} \;+\; Q \cdot \bar c_{\text{tok}}, \qquad \text{with} \quad M_{\text{resident}} \le B_{\text{mem}}. Compression pays mostly CreuseC_{\text{reuse}} , a documented small fraction of a from-scratch training cost [ 3 ] , but only if a suitable large model is available to reuse in the first place. Training small on purpose pays CcurateC_{\text{curate}} and CtrainC_{\text{train}} in full, with no discount, in exchange for a model that inherits nothing it was not deliberately given.…

Read this part in the article →

Learn the underlying idea

A subscript is a label attached below a symbol. It often selects a time step, component, category, or member of a sequence.

Open the illustrated subscripts: which member of a family? guide →

Sources cited in the surrounding passage

These citations provide research context; check each source for the exact claim it supports.