multiplication

The passage around this formula

The energy version of the same expression is more useful still. If e is the average energy per byte moved off-chip, energy per token is roughly e $\cdot$ B/b — the weight traffic divided across the batch, plus each request’s own cache traffic. Batching therefore improves energy per token steeply at first and then asymptotically, converging on the irreducible per-request cache cost. That asymptote is the floor referred to throughout this article, and only cache-reduction techniques lower it.

Read this part in the article →