Symbol E

The passage around this formula

The size of the win has a clean shape. Model the draft’s acceptance probability as $\alpha$ per token, roughly constant and independent across the $\gamma$ tokens proposed in a round — an idealization real traffic does not fully satisfy, but a useful one for seeing the ceiling. The expected number of tokens accepted per verification round is then $\mathbb{E}[\text{tokens per round}] = \frac{1-\alpha^{\gamma+1}}{1-\alpha}$. This rises with both $\alpha$ and $\gamma$ , but with steeply diminishing returns in $\gamma$ for any $\alpha$ below one: drafting fifty tokens ahead does not buy anywhere near fifty accepted tokens, because the marginal proposal deep into a long draft is unlikely to be exactly what the target would have generated. The bottleneck decode…

Read this part in the article →