What does this equation mean?

Large language model agents are not deterministic in practice, even holding the prompt and the environment fixed, and this is why the tool-use literature evaluates repeated trials rather than single runs. τ-bench formalizes the distinction with two related quantities. If a single attempt succeeds with probability p , and attempts are treated as independent, $\text{pass@}k = 1-(1-p)^k, \qquad \text{pass}^k = p^k$. where pass@ k is the probability that at least one of k attempts succeeds, and pass ^k is the probability that all k succeed [ 3 ] . The two statistics move in opposite directions as k grows: pass@ k climbs toward certainty, which is the right question when a system can retry until something works or a human picks the best of several drafts, while pass ^k falls toward zero, which is the right question for an agent deployed without a human standing by to catch the failures. Yao and colleagues report that state-of-the-art function-calling agents solved under half of τ-bench’s tasks on a single attempt, and that their pass ^k scores fell substantially with repeated trials of the same task under identical starting conditions, exposing an inconsistency that a single pass rate could not reveal on its own [ 3 ] .