← Back to article

Equation 16 · The Main Technical Approaches to AI Alignment, Compared

What does this equation mean?

Xt+1  =  Distill(AmplifyH(Xt)),X_{t+1} \;=\; \mathrm{Distill}\big(\mathrm{Amplify}_H(X_t)\big),

Read the formula alongside the article passage below. Each part has a deeper page with its role in the equation, the supporting passage and nearby citations.

Inputs and operationsDistillbig(Amplify_H(X_t)big)
Result or conditionX_t+1
How to read the two sides of this formula. Follow the article passage for the meaning of each quantity.

This equation states an equality: the expressions on both sides have the same value under the article’s assumptions. Read the equation part by part below; each part has a contextual explanation and a link to its mathematical background.

Read it piece by piece

Xt+1X_{t+1}

Symbol X_t+1

XtX_t+1 is the next indexed value computed from the current indexed quantity and the update terms shown on the right.

Understand this part →

HH

Symbol H

H is an input to the expression that computes the quantity on the left.

Understand this part →

XtX_t

Symbol X_t

XtX_t is an input to the expression that computes the quantity on the left.

Understand this part →

=

=

The expressions on both sides represent the same quantity under the stated assumptions.

Understand this part →

See an illustrated explanation →
addition

addition

Add the term after the plus sign to the term or group before it.

Understand this part →

subscript

subscript

The lower label selects a particular version, component, or indexed member of the quantity. For example, x₀ and xₜ can be values at different positions.

Understand this part →

How to interpret it

Read it with the definitions, units, and assumptions supplied by the article.

What the article says around this equation

IDA is debate’s closest theoretical relative, aimed at the same underlying worry from a different angle: rather than trusting a single judge to adjudicate an argument about a hard problem, it never asks any one evaluator to face the hard problem directly. Christiano, Shlegeris, and Amodei describe a scheme they call Amplify: a composite system, written AmplifyH(X)\mathrm{Amplify}_H(X) , “consisting of H and several copies of X working together to solve a problem,” in which H — standing in for a human overseer — breaks a hard question into useful subquestions, has several copies of the current trained agent X answer each subquestion, and then combines the subanswers into a response to the original…
Read the full surrounding passage
IDA is debate’s closest theoretical relative, aimed at the same underlying worry from a different angle: rather than trusting a single judge to adjudicate an argument about a hard problem, it never asks any one evaluator to face the hard problem directly. Christiano, Shlegeris, and Amodei describe a scheme they call Amplify: a composite system, written AmplifyH(X)\mathrm{Amplify}_H(X) , “consisting of H and several copies of X working together to solve a problem,” in which H — standing in for a human overseer — breaks a hard question into useful subquestions, has several copies of the current trained agent X answer each subquestion, and then combines the subanswers into a response to the original question [ 11 ] . The trained agent is then updated to imitate that amplified process directly: “the agent X then learns from AmplifyH(X)\mathrm{Amplify}_H(X) in the same way that it would traditionally learn from H alone” [ 11 ] . Run repeatedly, the described process is naturally read as the recursion Xt+1  =  Distill(AmplifyH(Xt))X_{t+1} \;=\; \mathrm{Distill}\big(\mathrm{Amplify}_H(X_t)\big). with each iteration’s agent trained to imitate the amplified, decomposed judgment of the previous iteration rather than any single overseer’s raw judgment of the whole problem. The paper describes the intended trajectory in prose rather than in this compact notation, but the substance is the same: over successive iterations, “the agent X becomes more powerful and the role of the expert transitions into ‘coordinating’ several copies of X to solve the problem better than a single copy could solve it” [ 11 ] .

Read the equation in its article →

Sources cited in the surrounding passage

These citations give research context. Read each source to check which claims it supports.

Return to The Main Technical Approaches to AI Alignment, Compared

See this formula across 1 published context →

Browse the mathematical compendium →