$$\text{Judge}(a) = \begin{cases} \text{automated}, & \hat\epsilon(a) \le \epsilon_{\max}(a) \ \text{and}\ \hat\epsilon(a)\ \text{externally audited} \\ \text{human required}, & \text{otherwise} \end{cases}$$

Equation 4 · Model Evaluation

Agent Evaluation in 2035: Two Axes, Four Scenarios, and What Would Falsify Them

This equation states a bound: one expression must stay on the indicated side of the other under the article’s assumptions.

Whether automated LLM-judge evaluation becomes trusted for high-stakes decisions is not a third axis; it is mostly a readout of Axis A applied to one specific evaluator. A useful way to see the dependency is to write the automation decision as a threshold rule. Let $\hat\epsilon(a)$ be the estimated error rate of a judge on a decision class a , and let $\epsilon_{\max}(a)$ be the maximum error a policy is willing to tolerate for that stakes class. A defensible automation rule is $\text{Judge}(a) = \begin{cases} \text{automated}, & \hat\epsilon(a) \le \epsilon_{\max}(a) \ \text{and}\ \hat\epsilon(a)\ \text{externally audited} \\ \text{human required}, & \text{otherwise} \end{cases}$. The rule only licenses automation where both clauses hold, and the second clause is the one Axis A supplies or withholds. Zheng and colleagues’ own eighty-percent figure plausibly satisfies the first…

Equation guide → · Article →