$$s_{\mathrm{adj}} \approx s_{\mathrm{reported}} \times (1-\mathrm{fp})$$

Equation 14 · Model Evaluation

OpenAI and Claude on Agentic Coding: What the Independent Evidence Actually Shows

This equation gives an approximation: it relates the quantities while allowing an approximation.

The size of that drop is close to what a simple deflation model predicts. If $s_{\mathrm{reported}}$ is the score under the original oracle and $\mathrm{fp}$ is the false-positive rate the adversarial suite exposes, then $s_{\mathrm{adj}} \approx s_{\mathrm{reported}} \times (1-\mathrm{fp})$. gives 0.7880 $\times$ (1 - 0.1971) $\approx$ 0.633 — within a percentage point of the measured 0.622 [ 14 ] . That closeness is a coincidence of rounding as much as a proof of the model, since rejected patches are not independent of task difficulty, but the approximation is useful precisely because it names the assumption plainly: a benchmark score is a joint statement about the system under test and the strength of the judge grading it, and when the judge gets…

Meanings in this article

• $s_{\mathrm{reported}}$: the score under the original oracle and $\mathrm{fp}$ is the false-positive rate the adversarial suite exposes.

Equation guide → · Article →