Equation 13 · Part 8 · AI Memory Systems and the Bandwidth Wall in 2035: Scenarios, Signals, and Falsifiable Predictions
=
=
What this part means
The expressions on both sides represent the same quantity under the stated assumptions.
Its job in the formula
The equals sign connects the complete expression on the left with the complete expression on the right. Both sides must have compatible units.
Full expression→=→Article meaning
The passage around this formula
Fact. The memory a transformer must hold for one in-flight generation — its key-value cache — grows linearly in exactly the variables that make serving expensive: sequence length and batch size. For a model with L layers, key-value heads, head dimension , sequence length S , batch size B , and p bytes stored per element, the cache occupies approximately . bytes, the leading factor of two accounting for keys and values together. That equation is worth writing out because it exposes the one real lever every mitigation below actually pulls: none of them escapes linear growth in S and B . Each instead shrinks one of the other factors — most consequentially…
Learn the underlying idea
An equals sign says that the expression on its left and the expression on its right have the same value under the stated definitions and assumptions.
Open the illustrated equality: what the equals sign claims guide →
Sources cited in the article section
- [15] DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model ↗
- [14] Efficient Streaming Language Models with Attention Sinks ↗
- [16] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models ↗
These citations provide research context; check each source for the exact claim it supports.