$$Q^{*}_{\mathrm{distill}} = \frac{C_{\mathrm{distill}}}{c_{\mathrm{teacher}} - c_{\mathrm{student}}}$$

Equation 7 · Edge AI & Electronics

The Real Economics of Shipping a Model to a Device

This equation states an equality: the expressions on both sides have the same value under the article’s assumptions.

Here $C_{\mathrm{distill}}$ is the one-time training cost, Q is the number of queries served over the deployment’s life, and $c_{\mathrm{student}}$ and $c_{\mathrm{teacher}}$ are the marginal cost per query of serving each model, wherever that serving happens. The break-even query volume is $Q^{*}_{\mathrm{distill}} = \frac{C_{\mathrm{distill}}}{c_{\mathrm{teacher}} - c_{\mathrm{student}}}$. As a worked illustration only, not a claim about any real company’s ledger: take Alpaca’s disclosed $600 and assume, as an order-of-magnitude figure consistent with how frontier and compact API tiers are typically priced relative to each other, that a teacher-class model costs $0.002 per query to serve and a well-distilled student costs $0.0002 — a tenfold gap. Then $Q^{*}_{\mathrm{distill}}$…

Meanings in this article

• $Q^{*}_{\mathrm{distill}}$: the break-even query volume.
• $C_{\mathrm{distill}}$: the one-time training cost.
• $c_{\mathrm{teacher}}$: the marginal cost per query of serving each model, wherever that serving happens.

Equation guide → · Article →