Equation 1 · Part 3 · What Alignment Actually Costs

Symbol C_RL

C_{\mathrm{align}} \approx C_{\mathrm{human}} + C_{\mathrm{RL}} + C_{\mathrm{redteam}}

C_{\mathrm{RL}}

What this part means

the compute spent training a policy against a reward model or a verifier, priced in GPU-hours and bounded by the same hardware capability development competes for.

Its job in the formula

$C_R$ L is a part of this expression. Its role is fixed by the surrounding article and by the operations shown in the formula.

Full expressionSymbol C_RLArticle meaning

Where the article explains it

$C_{\mathrm{RL}}$ is the compute spent training a policy against a reward model or a verifier, priced in GPU-hours and bounded by the same hardware capability development competes for.

The passage around this formula

…of scarcity: $C_{\mathrm{align}} \approx C_{\mathrm{human}} + C_{\mathrm{RL}} + C_{\mathrm{redteam}}$ . $C_{\mathrm{human}}$ is the cost of collecting human feedback — demonstrations and preference comparisons, priced per label and bounded by how many people can be recruited and trained. $C_{\mathrm{RL}}$ is the compute spent training a policy against a reward model or a verifier, priced in GPU-hours and bounded by the same hardware capability development competes for. $C_{\mathrm{redteam}}$ is the cost of adversarially testing the result, priced in expert-hours and bounded by how many…

Read this part in the article →

Learn the underlying idea

A subscript is a label attached below a symbol. It often selects a time step, component, category, or member of a sequence.

Open the illustrated subscripts: which member of a family? guide →

See this notation across published equations →

The article lists its research sources here.