$$E_{q} = \bar{P} \cdot \frac{N_{\mathrm{tok}}}{r}$$

Equation 10 · Small & Edge Models

The Deployment Envelope: Small Models Where the Power Is Not

This equation states an equality: the expressions on both sides have the same value under the article’s assumptions.

Deliberation. This is the sharpest inversion in the whole subject. The most reliable modern route to harder reasoning is to spend more computation at inference time — more tokens, more attempts, more search. In a datacentre that compute is available on demand and is charged for. On a battery-powered, thermally capped device it is the single most expensive thing you can do, because energy per query scales with tokens generated: $E_{q} = \bar{P} \cdot \frac{N_{\mathrm{tok}}}{r}$. with mean power $\bar{P}$ , tokens generated $N_{\mathrm{tok}}$ and sustained token rate r . Both $\bar{P}$ and r are set by the thermal wall, and r falls as the device heats — so a long deliberation is charged at a worsening exchange rate the longer it…

Meanings in this article

• $N_{\mathrm{tok}}$: the tokens generated.
• $r$: set by the thermal wall, and r falls as the device heats — so a long deliberation is charged at a worsening exchange rate the longer it runs, which is exactly the behaviour the sustained-load measurements show [ 6 , 5 ].

Equation guide → · Article →