≤

The passage around this formula

Whether human-approval checkpoints become automated and statistical is, likewise, mostly a readout of Axis B rather than a genuinely separate question. A useful way to see the dependency is to write the automation decision as a threshold rule. Let p(a) be a certified upper bound on the probability that action a violates its specification, and let S(a) be a severity score for the consequence class a belongs to. A natural rule an organization or regulator could adopt is $\text{Approval}(a) = \begin{cases} \text{automatic}, & p(a)\, S(a) \le B \\ \text{human required}, & p(a)\, S(a) > B \end{cases}$. where B is a risk budget set by policy. The rule only does useful work where p(a) is a number someone can trust — which is exactly what Axis B’s certified branch would supply and its empirical branch would…

Read this part in the article →