Equation 27 · Part 1 · Measuring AI Agent Reliability: What the Evidence Actually Supports
Symbol k
What this part means
k is a part of this expression. Its role is fixed by the surrounding article and by the operations shown in the formula.
Its job in the formula
k is a part of this expression. Its role is fixed by the surrounding article and by the operations shown in the formula.
Full expression→Symbol k→Article meaning
The passage around this formula
A single successful run is evidence that a task is solvable, not evidence that a system solves it reliably, and the statistics that separate the two claims already exist: pass@k for whether at least one of several attempts succeeds, pass ^k for the much stricter question of whether every one of them does, confidence intervals for whether an observed difference is more than noise, time-horizon curves for how reliability changes as a task grows, and a clear accounting of how a number was elicited before it is compared against how a system actually behaves once deployed. None of this is a call for more skepticism in the abstract. It is a call to ask, of any reported agent capability, which of…
Learn the underlying idea
A variable is a named place for a value. Its letter is a local label: x can mean position in one formula and a data point in another.
Open the illustrated variables: a letter stands for a value guide →
See this notation across published equations →
The article lists its research sources here.