Equation 9 · Part 5 · A History of How We Learned to Evaluate AI Agents
=
=
What this part means
The expressions on both sides represent the same quantity under the stated assumptions.
Its job in the formula
The equals sign connects the complete expression on the left with the complete expression on the right. Both sides must have compatible units.
Full expression→=→Article meaning
The passage around this formula
That last detail forced a genuine statistical problem into agent-adjacent evaluation for the first time: naively estimating the chance that at least one of k sampled attempts succeeds, by drawing exactly k samples and checking, is a high-variance estimator, especially at small k . Chen and colleagues instead drew a larger fixed pool of n samples per problem, counted the number c that passed, and computed an unbiased estimate of the pass rate at budget k directly from that pool: . The term inside the brackets is the probability that a random draw of k items from the n samples contains no passing solution, so one minus that quantity is the probability at least one does. The…
Learn the underlying idea
An equals sign says that the expression on its left and the expression on its right have the same value under the stated definitions and assumptions.
Open the illustrated equality: what the equals sign claims guide →
Sources cited in the article section
These citations provide research context; check each source for the exact claim it supports.