← All parts of this equation

Equation 9 · Part 3 · A History of How We Learned to Evaluate AI Agents

Symbol n

pass@k  =  Eproblems ⁣[ 1−(n−ck)(nk) ].\text{pass@}k \;=\; \mathbb{E}_{\text{problems}}\!\left[\,1 - \frac{\binom{n-c}{k}}{\binom{n}{k}}\,\right].
nn

What this part means

the number of samples.

Its job in the formula

n appears inside an expected value, so its contribution is averaged under the distribution or condition shown by that operator.

Where the article explains it

Chen and colleagues instead drew a larger fixed pool of n samples per problem, counted the number c that passed, and computed an unbiased estimate of the pass rate at budget k directly from that pool: pass@k  =  Eproblems ⁣[ 1−(n−ck)(nk) ]\text{pass@}k \;=\; \mathbb{E}_{\text{problems}}\!\left[\,1 - \frac{\binom{n-c}{k}}{\binom{n}{k}}\,\right].

The passage around this formula

…the chance that at least one of k sampled attempts succeeds, by drawing exactly k samples and checking, is a high-variance estimator, especially at small k . Chen and colleagues instead drew a larger fixed pool of n samples per problem, counted the number c that passed, and computed an unbiased estimate of the pass rate at budget k directly from that pool: pass@k  =  Eproblems ⁣[ 1−(n−ck)(nk) ]\text{pass@}k \;=\; \mathbb{E}_{\text{problems}}\!\left[\,1 - \frac{\binom{n-c}{k}}{\binom{n}{k}}\,\right]. The term inside the brackets is the probability that a random draw of k items from the n samples contains no passing solution, so one…

Read this part in the article →

Learn the underlying idea

A variable is a named place for a value. Its letter is a local label: x can mean position in one formula and a data point in another.

Open the illustrated variables: a letter stands for a value guide →

See this notation across published equations →

Sources cited in the article section

These citations provide research context; check each source for the exact claim it supports.