addition

The passage around this formula

Start with the conventional mixed-effects baseline specified before task results are opened: $\operatorname{logit}\Pr(Y_{imtr}=1)= \beta_0+\alpha_m+\beta_1 q_{tr}+\beta_2 d_t+ \beta_3\log(1+h_t)+u_{\mathrm{repo}(t)}+u_t$. Here $q_{tr}$ is supplied input-context length in tokens, not tokens consumed after the agent has begun acting; using post-attempt expenditure would contaminate the predictor with behavior. The term $d_t$ is a predeclared ordinary benchmark-difficulty score based only on endpoint-side, redaction-invariant features such as repository size band, static dependency reach, test-suite scope, and selected task-family label. It is deliberately conventional and admittedly imperfect. The terms $u_{\mathrm{repo}(t)}$ and $u_t$ are repository and task random effects, while $\alpha_m$ represents model…

Read this part in the article →