Equation 13 · Part 11 · Serving a Frontier Model: The KV Cache, Batching, and What a Token Actually Costs
Denominator: M_kv(s)
What this part means
The complete quantity below the fraction bar; it must be nonzero for this division.
Its job in the formula
v(s) occurs below the fraction bar. The quantity above the bar is divided by this expression; zero is excluded as a denominator.
Full expression→Denominator: M_kv(s)→Article meaning
The passage around this formula
What binds is memory, and specifically the cache. Weights are shared across the batch; the key–value cache is not. Each concurrent request carries its own, and each grows with every token it generates. Achievable batch size is therefore . which falls as contexts lengthen. This is the mechanism behind an effect users notice without explanation: long-context workloads cost disproportionately more, because they crowd out the concurrency that made short-context serving cheap.
Learn the underlying idea
A fraction a/b means a divided by b. The top number is the numerator; the bottom number is the denominator, and it cannot be zero.
Open the illustrated fractions: division written vertically guide →
Sources cited in the article section
- [2] Efficiently Scaling Transformer Inference ↗
- [3] Efficient Memory Management for Large Language Model Serving with PagedAttention ↗
These citations provide research context; check each source for the exact claim it supports.