What does this equation mean?

Consider what the o1 numbers alone expose about how these gains are produced. If a single attempt at a problem succeeds with probability p , and k attempts were independent, the probability that at least one succeeds is $p_k = 1 - (1-p)^k$. a curve that climbs fast and then saturates. It is tempting to read o1’s jump from 74% (one sample) to 83% (64-sample consensus) to 93% (1,000-sample rerank) as roughly this shape. It is not, for two reasons that matter for how the number should be read. First, repeated attempts by one model on one problem are correlated — the same misconception that causes one failure tends to recur — so the realised gain from added samples falls well below what independence would predict. Second, the improvement from 83% to 93% is not sampling alone: it required “a learned scoring function” [ 1 ] to pick the right answer out of 1,000 candidates, which is itself a second trained system doing real work, not a free byproduct of asking the model to try more times. The same logic applies to Anthropic’s “internal scoring model” for selecting among parallel attempts [ 4 ] . In both cases, a benchmark score produced this way is a property of model plus sampling budget plus selector , not of the base model’s reasoning in isolation — which is exactly why both companies now report the configuration alongside the score rather than the score alone.