≈

The passage around this formula

The size of that drop is close to what a simple deflation model predicts. If $s_{\mathrm{reported}}$ is the score under the original oracle and $\mathrm{fp}$ is the false-positive rate the adversarial suite exposes, then $s_{\mathrm{adj}} \approx s_{\mathrm{reported}} \times (1-\mathrm{fp})$. gives 0.7880 $\times$ (1 - 0.1971) $\approx$ 0.633 — within a percentage point of the measured 0.622 [ 14 ] . That closeness is a coincidence of rounding as much as a proof of the model, since rejected patches are not independent of task difficulty, but the approximation is useful precisely because it names the assumption plainly: a benchmark score is a joint statement about the system under test and the strength of the judge grading it, and when the judge gets…

Read this part in the article →