=

The passage around this formula

That range is worth writing as a simple decomposition, because it clarifies exactly what a published score is actually a measurement of: $s = g(\theta,\; H,\; e,\; \tau,\; \phi)$. where $\theta$ is the model’s weights, H is the harness or scaffold wrapped around it, e is the reasoning-effort or thinking-budget setting, $\tau$ is the strength of the test oracle used to grade the output, and $\phi$ is the model’s likely prior exposure to the benchmark’s specific tasks during training. A score gap between two systems is informative about $\theta$ — the thing “OpenAI versus Claude” is supposed to mean — only when H , e , $\tau$ , and $\phi$ are held fixed across both measurements. The evidence above shows that, on the…

Read this part in the article →