=

The passage around this formula

Deliberation. This is the sharpest inversion in the whole subject. The most reliable modern route to harder reasoning is to spend more computation at inference time — more tokens, more attempts, more search. In a datacentre that compute is available on demand and is charged for. On a battery-powered, thermally capped device it is the single most expensive thing you can do, because energy per query scales with tokens generated: $E_{q} = \bar{P} \cdot \frac{N_{\mathrm{tok}}}{r}$. with mean power $\bar{P}$ , tokens generated $N_{\mathrm{tok}}$ and sustained token rate r . Both $\bar{P}$ and r are set by the thermal wall, and r falls as the device heats — so a long deliberation is charged at a worsening exchange rate the longer it…

Read this part in the article →