Equation 10 · Part 1 · A History of How We Learned to Evaluate AI Agents
Symbol k
What this part means
the especially at small.
Its job in the formula
k is a part of this expression. Its role is fixed by the surrounding article and by the operations shown in the formula.
Full expression→Symbol k→Article meaning
Where the article explains it
That last detail forced a genuine statistical problem into agent-adjacent evaluation for the first time: naively estimating the chance that at least one of k sampled attempts succeeds, by drawing exactly k samples and checking, is a high-variance estimator, especially at small k .
The passage around this formula
The term inside the brackets is the probability that a random draw of k items from the n samples contains no passing solution, so one minus that quantity is the probability at least one does. The point of writing the estimator this way, rather than simply sampling k times per problem, is to separate two things later agent benchmarks would have to separate again and again: how good a system is, and how much it was allowed to try. Every agent benchmark discussed below that reports a success rate is implicitly answering the question this estimator first made explicit — success at what sampling budget, counted how.
Learn the underlying idea
A variable is a named place for a value. Its letter is a local label: x can mean position in one formula and a data point in another.
Open the illustrated variables: a letter stands for a value guide →
See this notation across published equations →
Sources cited in the article section
These citations provide research context; check each source for the exact claim it supports.