Equation 3 · Part 4 · The Hardest Unsolved Problems in Frontier AI Model Comparisons

Symbol delta_i,b

\hat{s}_{i,b} = g_b(\theta_i) + \delta_{i,b} + \varepsilon_{i,b},

\delta_{i,b}

What this part means

a contamination or leakage term specific to that model-benchmark pair.

Its job in the formula

delt $a_i$ ,b is one of the signed contributions combined to compute the quantity on the left.

Full expressionSymbol delta_i,bArticle meaning

Where the article explains it

where $\theta_i$ is some unobservable underlying ability vector for model i , $g_b$ is the mapping from ability to expected score that is specific to benchmark b and not shared across benchmarks, $\delta_{i,b}$ is a contamination or leakage term specific to that model-benchmark pair, and $\varepsilon_{i,b}$ is sampling and decoding noise.

The passage around this formula

…as $\hat{s}_{i,b} = g_b(\theta_i) + \delta_{i,b} + \varepsilon_{i,b}$ . where $\theta_i$ is some unobservable underlying ability vector for model i , $g_b$ is the mapping from ability to expected score that is specific to benchmark b and not shared across benchmarks, $\delta_{i,b}$ is a contamination or leakage term specific to that model-benchmark pair, and $\varepsilon_{i,b}$ is sampling and decoding noise. Because $g_b$ differs across benchmarks by construction — a multiple-choice knowledge test and a pairwise human-preference vote are not measuring the same…

Read this part in the article →

Learn the underlying idea

A subscript is a label attached below a symbol. It often selects a time step, component, category, or member of a sequence.

Open the illustrated subscripts: which member of a family? guide →

See this notation across published equations →

Sources cited in the article section

These citations provide research context; check each source for the exact claim it supports.