← All parts of this equation

Equation 2 · Part 3 · The Hardest Unsolved Problems in AI Agent Evaluation and Reliability

Symbol p^k

pass@k=1−(1−p)k,passk=pk,\text{pass@}k = 1-(1-p)^k, \qquad \text{pass}^k = p^k,
pkp^k

What this part means

pkp^k is one of the signed contributions combined to compute the quantity on the left.

Its job in the formula

pkp^k is one of the signed contributions combined to compute the quantity on the left.

The passage around this formula

Large language model agents are not deterministic in practice, even holding the prompt and the environment fixed, and this is why the tool-use literature evaluates repeated trials rather than single runs. τ-bench formalizes the distinction with two related quantities. If a single attempt succeeds with probability p , and attempts are treated as independent, pass@k=1−(1−p)k,passk=pk\text{pass@}k = 1-(1-p)^k, \qquad \text{pass}^k = p^k. where pass@ k is the probability that at least one of k attempts succeeds, and pass ^k is the probability that all k succeed [ 3 ] . The two statistics move in opposite directions as k grows: pass@ k climbs toward certainty, which is the right question when a system can retry until something works or a human picks the…

Read this part in the article →

Learn the underlying idea

An exponent tells how a base is used in multiplication. In x³, x is the base and 3 is the exponent: x³ = x × x × x.

Open the illustrated exponents: repeated multiplication and powers guide →

See this notation across published equations →

Sources cited in the surrounding passage

These citations provide research context; check each source for the exact claim it supports.