Equation 4 · Part 3 · The Physical Economics of Inference: Bandwidth, Energy, and the Limits of a Datacenter

Symbol n_kv

M_{\mathrm{kv}} = 2\,L\,n_{\mathrm{kv}}\,d_{\mathrm{head}}\,s\,p,

n_{\mathrm{kv}}

What this part means

$n_k$ v is an input to the expression that computes the quantity on the left.

Its job in the formula

$n_k$ v is an input to the expression that computes the quantity on the left.

Full expressionSymbol n_kvArticle meaning

The passage around this formula

Take a dense model with N parameters served at p bytes per parameter. The weight traffic per decoding step is Np , incurred once per step regardless of how many requests share it. The per-request cache traffic is $M_{\mathrm{kv}} = 2\,L\,n_{\mathrm{kv}}\,d_{\mathrm{head}}\,s\,p$ . read in full every step for every active request. Total bytes moved per step across a batch of b requests at sequence length s is therefore approximately

Read this part in the article →

Learn the underlying idea

A subscript is a label attached below a symbol. It often selects a time step, component, category, or member of a sequence.

Open the illustrated subscripts: which member of a family? guide →

See this notation across published equations →

The article lists its research sources here.