Symbol k
k is a part of this expression. Its role is fixed by the surrounding article and by the operations shown in the formula.
Read this term in its guide →Published equation contexts
Why this formula appears here
averaged over tasks to produce the benchmark’s headline number. The metric is designed to punish an agent whose competence is real but inconsistent across resampled trials — a stochastic policy with true per-trial success probability p has , which falls quickly as k grows. But a policy whose output on a given task is deterministic — an empty response, always, regardless of sampling — produces the identical transcript on every trial, so
Read the representative guide
Symbol p^k
is a part of this expression. Its role is fixed by the surrounding article and by the operations shown in the formula.
Read this term in its guide →How to interpret it
Its accuracy depends on the assumptions and range of use described in the article.
Research cited beside this formula
Published contexts (1)
A symbol can carry a different meaning in another article. Each occurrence keeps its own guide and term definitions.
Equation 10 · Model Evaluation
How Benchmark Contamination Actually Works in Agentic Evaluation
This equation gives an approximation: it relates the quantities while allowing an approximation.
averaged over tasks to produce the benchmark’s headline number. The metric is designed to punish an agent whose competence is real but inconsistent across resampled trials — a stochastic policy with true per-trial success probability p has , which falls quickly as k grows. But a policy whose output on a given task is deterministic — an empty response, always, regardless of sampling — produces the identical transcript on every trial, so
Equation guide → · Article →