Symbol t_token
oken is a part of this expression. Its role is fixed by the surrounding article and by the operations shown in the formula.
Read this term in its guide →Published equation contexts
Why this formula appears here
Underneath all of the pricing mechanics sits a constraint pricing cannot remove: generating one token from a large language model is, for most of a request, bound by how fast bytes can move through accelerator memory rather than by how much arithmetic the accelerator can perform. Producing each token requires reading the model’s active weights and the accumulated key-value cache for that request from memory once; the arithmetic performed per byte read is low, so decoding is memory-bandwidth-bound while only the initial processing of a long prompt is compute-bound, a distinction Pope and colleagues formalized in their analysis of transformer inference efficiency [ 15 ] . Approximately,…
Read the representative guide
Symbol N_active
ctive occurs above the fraction bar. The numerator is divided by the entire denominator below it.
Read this term in its guide →Numerator: 2N_activeβ + KV(c)
The complete quantity above the fraction bar.
Read this term in its guide →Denominator: BW
The complete quantity below the fraction bar; it must be nonzero for this division.
Read this term in its guide →How to interpret it
With a fixed numerator, increasing a nonzero denominator reduces the fraction.
Research cited beside this formula
Published contexts (1)
A symbol can carry a different meaning in another article. Each occurrence keeps its own guide and term definitions.
Equation 2 · AI Agents & Systems
The Economics, Energy, and Physical Limits of Claude Code and Agentic Development Tools
This mathematical expression combines the displayed quantities; its precise role follows from the surrounding article text.
Underneath all of the pricing mechanics sits a constraint pricing cannot remove: generating one token from a large language model is, for most of a request, bound by how fast bytes can move through accelerator memory rather than by how much arithmetic the accelerator can perform. Producing each token requires reading the model’s active weights and the accumulated key-value cache for that request from memory once; the arithmetic performed per byte read is low, so decoding is memory-bandwidth-bound while only the initial processing of a long prompt is compute-bound, a distinction Pope and colleagues formalized in their analysis of transformer inference efficiency [ 15 ] . Approximately,…