Symbol e
the average energy per byte moved off-chip, energy per token is roughly e B/b — the weight traffic divided across the batch, plus each request’s own cache traffic.
Read this term in its guide →Published equation contexts
Why this formula appears here
The energy version of the same expression is more useful still. If e is the average energy per byte moved off-chip, energy per token is roughly e B/b — the weight traffic divided across the batch, plus each request’s own cache traffic. Batching therefore improves energy per token steeply at first and then asymptotically, converging on the irreducible per-request cache cost. That asymptote is the floor referred to throughout this article, and only cache-reduction techniques lower it.
Read the representative guide
Symbol B
B is a part of this expression. Its role is fixed by the surrounding article and by the operations shown in the formula.
Read this term in its guide →Symbol b
b is a part of this expression. Its role is fixed by the surrounding article and by the operations shown in the formula.
Read this term in its guide →How to interpret it
Read this expression with the definitions, units, and assumptions supplied by the article.
Published contexts (1)
A symbol can carry a different meaning in another article. Each occurrence keeps its own guide and term definitions.
Equation 12 · Foundation Models
The Physical Economics of Inference: Bandwidth, Energy, and the Limits of a Datacenter
This mathematical expression combines the displayed quantities; its precise role follows from the surrounding article text.
The energy version of the same expression is more useful still. If e is the average energy per byte moved off-chip, energy per token is roughly e B/b — the weight traffic divided across the batch, plus each request’s own cache traffic. Batching therefore improves energy per token steeply at first and then asymptotically, converging on the irreducible per-request cache cost. That asymptote is the floor referred to throughout this article, and only cache-reduction techniques lower it.
Meanings in this article
- : the average energy per byte moved off-chip, energy per token is roughly e B/b — the weight traffic divided across the batch, plus each request’s own cache traffic.