← All parts of this equation

Equation 12 · Part 2 · How Benchmark Contamination Actually Works in Agentic Evaluation

Symbol k

passdeterministick=pass1for every k.\text{pass}^k_{\text{deterministic}} = \text{pass}^1 \quad \text{for every } k .
kk

What this part means

k is part of the quantity the equation computes from the expression on the right.

Its job in the formula

k is part of the quantity the equation computes from the expression on the right.

The passage around this formula

…is designed to punish an agent whose competence is real but inconsistent across resampled trials — a stochastic policy with true per-trial success probability p has passk\text{pass}^k ≈\approx pkp^k , which falls quickly as k grows. But a policy whose output on a given task is deterministic — an empty response, always, regardless of sampling — produces the identical transcript on every trial, so passdeterministick=pass1for every k\text{pass}^k_{\text{deterministic}} = \text{pass}^1 \quad \text{for every } k . A degenerate exploit is not merely invisible to pass@1; it is more invisible to passks^k, precisely…

Read this part in the article →

Learn the underlying idea

A variable is a named place for a value. Its letter is a local label: x can mean position in one formula and a data point in another.

Open the illustrated variables: a letter stands for a value guide →

See this notation across published equations →

Sources cited in the article section

These citations provide research context; check each source for the exact claim it supports.