← Mathematical compendium

Published equation contexts

Mkv=2 L nkv dhead s pM_{\mathrm{kv}} = 2\,L\,n_{\mathrm{kv}}\,d_{\mathrm{head}}\,s\,p

Why this formula appears here

Take a dense model with N parameters served at p bytes per parameter. The weight traffic per decoding step is Np , incurred once per step regardless of how many requests share it. The per-request cache traffic is Mkv=2 L nkv dhead s pM_{\mathrm{kv}} = 2\,L\,n_{\mathrm{kv}}\,d_{\mathrm{head}}\,s\,p. read in full every step for every active request. Total bytes moved per step across a batch of b requests at sequence length s is therefore approximately

Read the full article-specific guide →

Read the representative guide

dheadd_{\mathrm{head}}

Symbol d_head

dhd_head is an input to the expression that computes the quantity on the left.

Read this term in its guide →

How to interpret it

Read it with the definitions, units, and assumptions supplied by the article.

Learn the mathematics behind it

Published contexts (1)

A symbol can carry a different meaning in another article. Each occurrence keeps its own guide and term definitions.

Mkv=2 L nkv dhead s p,M_{\mathrm{kv}} = 2\,L\,n_{\mathrm{kv}}\,d_{\mathrm{head}}\,s\,p,

Equation 4 · Foundation Models

The Physical Economics of Inference: Bandwidth, Energy, and the Limits of a Datacenter

This equation states an equality: the expressions on both sides have the same value under the article’s assumptions.

Take a dense model with N parameters served at p bytes per parameter. The weight traffic per decoding step is Np , incurred once per step regardless of how many requests share it. The per-request cache traffic is Mkv=2 L nkv dhead s pM_{\mathrm{kv}} = 2\,L\,n_{\mathrm{kv}}\,d_{\mathrm{head}}\,s\,p. read in full every step for every active request. Total bytes moved per step across a batch of b requests at sequence length s is therefore approximately

Meanings in this article

Equation guide → · Article →