Equation 16 · The Main Technical Approaches to AI Alignment, Compared
What does this equation mean?
Read the formula alongside the article passage below. Each part has a deeper page with its role in the equation, the supporting passage and nearby citations.
This equation states an equality: the expressions on both sides have the same value under the article’s assumptions. Read the equation part by part below; each part has a contextual explanation and a link to its mathematical background.
Read it piece by piece
Symbol X_t+1
+1 is the next indexed value computed from the current indexed quantity and the update terms shown on the right.
Symbol H
H is an input to the expression that computes the quantity on the left.
Symbol X_t
is an input to the expression that computes the quantity on the left.
=
The expressions on both sides represent the same quantity under the stated assumptions.
See an illustrated explanation →subscript
The lower label selects a particular version, component, or indexed member of the quantity. For example, x₀ and xₜ can be values at different positions.
How to interpret it
Read it with the definitions, units, and assumptions supplied by the article.
What the article says around this equation
IDA is debate’s closest theoretical relative, aimed at the same underlying worry from a different angle: rather than trusting a single judge to adjudicate an argument about a hard problem, it never asks any one evaluator to face the hard problem directly. Christiano, Shlegeris, and Amodei describe a scheme they call Amplify: a composite system, written , “consisting of H and several copies of X working together to solve a problem,” in which H — standing in for a human overseer — breaks a hard question into useful subquestions, has several copies of the current trained agent X answer each subquestion, and then combines the subanswers into a response to the original…
Read the full surrounding passage
IDA is debate’s closest theoretical relative, aimed at the same underlying worry from a different angle: rather than trusting a single judge to adjudicate an argument about a hard problem, it never asks any one evaluator to face the hard problem directly. Christiano, Shlegeris, and Amodei describe a scheme they call Amplify: a composite system, written , “consisting of H and several copies of X working together to solve a problem,” in which H — standing in for a human overseer — breaks a hard question into useful subquestions, has several copies of the current trained agent X answer each subquestion, and then combines the subanswers into a response to the original question [ 11 ] . The trained agent is then updated to imitate that amplified process directly: “the agent X then learns from in the same way that it would traditionally learn from H alone” [ 11 ] . Run repeatedly, the described process is naturally read as the recursion . with each iteration’s agent trained to imitate the amplified, decomposed judgment of the previous iteration rather than any single overseer’s raw judgment of the whole problem. The paper describes the intended trajectory in prose rather than in this compact notation, but the substance is the same: over successive iterations, “the agent X becomes more powerful and the role of the expert transitions into ‘coordinating’ several copies of X to solve the problem better than a single copy could solve it” [ 11 ] .
Sources cited in the surrounding passage
These citations give research context. Read each source to check which claims it supports.
Return to The Main Technical Approaches to AI Alignment, Compared