≈

The passage around this formula

As a worked illustration only, not a claim about any real company’s ledger: take Alpaca’s disclosed $600 and assume, as an order-of-magnitude figure consistent with how frontier and compact API tiers are typically priced relative to each other, that a teacher-class model costs $0.002 per query to serve and a well-distilled student costs $0.0002 — a tenfold gap. Then $Q^{*}_{\mathrm{distill}}$ = 600 / 0.0018 $\approx$ 333{,}000 queries. Any production feature serving that many requests clears the investment in days, sometimes hours. This is the arithmetic reason distillation has become the default route to a deployable small model rather than a research curiosity: the pipeline cost is a rounding…

Read this part in the article →