subtraction

The passage around this formula

More fundamentally, even a perfectly executed, continuously updated tracker cannot make the comparison one-dimensional, because quality at a given cost and latency is a function of a whole vector of settings — reasoning effort, context length used, modality, batching — and two systems compared at one point in that vector can reverse their relative standing at another point: $q_A(v) \gtrless q_B(v), \quad \text{where the sign can flip as the operating-point vector } v \text{ varies}$. A model that leads at high cost and maximum reasoning effort is not guaranteed to lead at the low-cost, low-latency point a production deployment actually needs, and a comparison that quotes only one of those two points without saying which is not wrong so much as silently incomplete. No published…

Read this part in the article →