Equation 7 · The Deployment Envelope: Small Models Where the Power Is Not
What does this equation mean?
Read the formula alongside the article passage below. Each part has a deeper page with its role in the equation, the supporting passage and nearby citations.
the achievable memory bandwidth and I the operational intensity of the kernel. Read the equation part by part below; each part has a contextual explanation and a link to its mathematical background.
Read it piece by piece
Symbol B
the achievable memory bandwidth and I the operational intensity of the kernel.
How to interpret it
Read this expression with the definitions, units, and assumptions supplied by the article.
What the article says around this equation
Autoregressive decoding sits at the far left of that graph. Generating one token requires reading essentially every weight and the accumulated key-value cache once, and performing roughly two arithmetic operations per parameter. At one byte per weight the intensity is about two operations per byte; at four bits per weight, about four. Pope and colleagues formalised this partitioning problem for large transformers and showed how latency, throughput and cost trade off under different sharding strategies, with generation and prefill behaving as different regimes [ 3 ] . The bound that matters on a device follows immediately: if W bytes of weights and cache must cross the memory bus for each…
Read the full surrounding passage
Autoregressive decoding sits at the far left of that graph. Generating one token requires reading essentially every weight and the accumulated key-value cache once, and performing roughly two arithmetic operations per parameter. At one byte per weight the intensity is about two operations per byte; at four bits per weight, about four. Pope and colleagues formalised this partitioning problem for large transformers and showed how latency, throughput and cost trade off under different sharding strategies, with generation and prefill behaving as different regimes [ 3 ] . The bound that matters on a device follows immediately: if W bytes of weights and cache must cross the memory bus for each token and the sustained bandwidth is B , then the time per token cannot be less than
Sources cited in the surrounding passage
These citations give research context. Read each source to check which claims it supports.
Return to The Deployment Envelope: Small Models Where the Power Is Not