← All parts of this equation

Equation 15 · Part 9 · The Hardest Unsolved Problems in AI Agent Evaluation and Reliability

superscript

R=1−∑i=1nwi⋅1[trial i failed],R = 1 - \sum_{i=1}^{n} w_i \cdot \mathbb{1}[\text{trial } i \text{ failed}],
superscript

What this part means

A raised number can be a power. When it is a label or bound, it selects a case or the upper limit of a sum; the formula’s structure distinguishes these uses.

Its job in the formula

A raised mark can be a power or an index. Its position and the surrounding notation determine which.

The passage around this formula

and contrast it with a severity-weighted version, R=1−∑i=1nwi⋅1[trial i failed]R = 1 - \sum_{i=1}^{n} w_i \cdot \mathbb{1}[\text{trial } i \text{ failed}]. where wiw_i scales each failure by how costly it actually was. Two agents can share an identical p^\hat{p} of, say, ninety-five percent, while one of them fails harmlessly - an unhelpful but reversible answer - and the other fails catastrophically - an irreversible transaction, a deleted repository, a wrong medical dosage recommendation - on that same five percent. No standard agent benchmark publishes R , because assigning a defensible wiw_i requires a judgment about real-world consequence that a replayable, sandboxed task suite is not built to carry, and because the tasks that would carry the highest weights are, not…

Read this part in the article →

Learn the underlying idea

An exponent tells how a base is used in multiplication. In x³, x is the base and 3 is the exponent: x³ = x × x × x.

Open the illustrated exponents: repeated multiplication and powers guide →

Sources cited in the article section

These citations provide research context; check each source for the exact claim it supports.