← Mathematical compendium

Published equation contexts

n=100n=100

Why this formula appears here

That combinatorial form is the unbiased estimator: it uses every one of the n observed outcomes rather than discarding some of them, and it has materially lower variance than the intuitive shortcut of raising a single measured success rate to a power. The paper is explicit that the naive alternative is biased, not merely noisier, so the difference is not a technicality — using the wrong formula changes which system looks better [ 2 ] . With n=100 samples per problem, their Codex model solved 70.2% of HumanEval problems by this best-of-many-samples criterion, against 28.8% at pass@1 [ 2 ] . That gap is not a measurement error. It is the honest size of the difference between “can this model…

Read the full article-specific guide →

Read the representative guide

How to interpret it

Read it with the definitions, units, and assumptions supplied by the article.

Research cited beside this formula

Published contexts (1)

A symbol can carry a different meaning in another article. Each occurrence keeps its own guide and term definitions.

n=100n=100

Equation 9 · AI Agents & Systems

Measuring AI Agent Reliability: What the Evidence Actually Supports

This equation states an equality: the expressions on both sides have the same value under the article’s assumptions.

That combinatorial form is the unbiased estimator: it uses every one of the n observed outcomes rather than discarding some of them, and it has materially lower variance than the intuitive shortcut of raising a single measured success rate to a power. The paper is explicit that the naive alternative is biased, not merely noisier, so the difference is not a technicality — using the wrong formula changes which system looks better [ 2 ] . With n=100 samples per problem, their Codex model solved 70.2% of HumanEval problems by this best-of-many-samples criterion, against 28.8% at pass@1 [ 2 ] . That gap is not a measurement error. It is the honest size of the difference between “can this model…

Equation guide → · Article →