Equation 12 · Part 1 · AI Memory Systems and the Bandwidth Wall in Practice: An Advanced Technical Guide
Symbol H
What this part means
H is a part of this expression. Its role is fixed by the surrounding article and by the operations shown in the formula.
Its job in the formula
H is a part of this expression. Its role is fixed by the surrounding article and by the operations shown in the formula.
Full expression→Symbol H→Article meaning
The passage around this formula
The two terms interact because the KV cache and the batch draw on the same pool. Raising batch size raises b in the capacity equation directly; every request added to a batch adds its own cache, and past some point the node runs out of room for cache before it runs out of arithmetic. Multi-query and grouped-query attention relieve this by shrinking H , the number of distinct key-value heads a token’s cache must store — Pope and colleagues report that the resulting smaller footprint enables scaling to context lengths up to 32 times longer at fixed memory [ 7 ] , and the same shrinkage buys headroom for a larger batch instead, if that is the more valuable trade for a given service.
Learn the underlying idea
A variable is a named place for a value. Its letter is a local label: x can mean position in one formula and a data point in another.
Open the illustrated variables: a letter stands for a value guide →
See this notation across published equations →
Sources cited in the surrounding passage
These citations provide research context; check each source for the exact claim it supports.