Equation 9 · AI Memory Systems and the Bandwidth Wall in Practice: An Advanced Technical Guide
What does this equation mean?
Read the formula alongside the article passage below. Each part has a deeper page with its role in the equation, the supporting passage and nearby citations.
This mathematical expression combines the displayed quantities; its precise role follows from the surrounding article text. Read the equation part by part below; each part has a contextual explanation and a link to its mathematical background.
Read it piece by piece
Symbol s
s is a part of this expression. Its role is fixed by the surrounding article and by the operations shown in the formula.
How to interpret it
Read this expression with the definitions, units, and assumptions supplied by the article.
What the article says around this equation
Eviction attacks s directly by refusing to keep the whole history. The common pattern — a retained sliding window of recent tokens plus a small set of early “sink” tokens that attention disproportionately depends on — trades a bounded, workload-dependent quality cost for a hard, predictable ceiling on cache size regardless of how long a conversation runs. It is the right tool when a hard memory ceiling matters more than perfect recall of everything a request has ever said, and the wrong one when a downstream task genuinely depends on distant context; which regime a given product is in is an empirical question specific to that product’s workload, not a general property of eviction as a…
Read the full surrounding passage
Eviction attacks s directly by refusing to keep the whole history. The common pattern — a retained sliding window of recent tokens plus a small set of early “sink” tokens that attention disproportionately depends on — trades a bounded, workload-dependent quality cost for a hard, predictable ceiling on cache size regardless of how long a conversation runs. It is the right tool when a hard memory ceiling matters more than perfect recall of everything a request has ever said, and the wrong one when a downstream task genuinely depends on distant context; which regime a given product is in is an empirical question specific to that product’s workload, not a general property of eviction as a technique.
Sources cited in the article section
- [4] Efficient Memory Management for Large Language Model Serving with PagedAttention ↗
- [5] KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache ↗
These citations give research context. Read each source to check which claims it supports.
Return to AI Memory Systems and the Bandwidth Wall in Practice: An Advanced Technical Guide