Equation 4 · The Physical Economics of Inference: Bandwidth, Energy, and the Limits of a Datacenter
What does this equation mean?
Read the formula alongside the article passage below. Each part has a deeper page with its role in the equation, the supporting passage and nearby citations.
This equation states an equality: the expressions on both sides have the same value under the article’s assumptions. Read the equation part by part below; each part has a contextual explanation and a link to its mathematical background.
Read it piece by piece
Symbol L
L is an input to the expression that computes the quantity on the left.
Symbol n_kv
v is an input to the expression that computes the quantity on the left.
Symbol d_head
ead is an input to the expression that computes the quantity on the left.
Symbol p
p is an input to the expression that computes the quantity on the left.
=
The expressions on both sides represent the same quantity under the stated assumptions.
See an illustrated explanation →subscript
The lower label selects a particular version, component, or indexed member of the quantity. For example, x₀ and xₜ can be values at different positions.
How to interpret it
Read it with the definitions, units, and assumptions supplied by the article.
What the article says around this equation
Take a dense model with N parameters served at p bytes per parameter. The weight traffic per decoding step is Np , incurred once per step regardless of how many requests share it. The per-request cache traffic is . read in full every step for every active request. Total bytes moved per step across a batch of b requests at sequence length s is therefore approximately
For background, read the article’s source list.
Return to The Physical Economics of Inference: Bandwidth, Energy, and the Limits of a Datacenter