Equation 6 · Evaluating Trajectories, Not Answers
What does this equation mean?
Read the formula alongside the article passage below. Each part has a deeper page with its role in the equation, the supporting passage and nearby citations.
This mathematical expression combines the displayed quantities; its precise role follows from the surrounding article text. Read the equation part by part below; each part has a contextual explanation and a link to its mathematical background.
Read it piece by piece
Symbol k
k is a part of this expression. Its role is fixed by the surrounding article and by the operations shown in the formula.
Symbol p^k
is a part of this expression. Its role is fixed by the surrounding article and by the operations shown in the formula.
subtraction
Subtract the following term or group from the preceding one. A leading minus marks a negative quantity.
superscript
A raised number can be a power. When it is a label or bound, it selects a case or the upper limit of a sum; the formula’s structure distinguishes these uses.
See an illustrated explanation →How to interpret it
Read this expression with the definitions, units, and assumptions supplied by the article.
What the article says around this equation
The two quantities point in opposite directions as sampling increases. With per-attempt success probability p and independent attempts, compare . The first rises toward one; the second falls toward zero. They are computed from the same runs and answer entirely different questions, and reporting the first while the deployment depends on the second is the most common quiet misrepresentation in agent evaluation.
Sources cited in the article section
- [4] τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains ↗
- [10] The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs ↗
- [9] Measuring AI Ability to Complete Long Software Tasks ↗
These citations give research context. Read each source to check which claims it supports.
Return to Evaluating Trajectories, Not Answers