Symbol y_w
is a part of this expression. Its role is fixed by the surrounding article and by the operations shown in the formula.
Read this term in its guide →Published equation contexts
Why this formula appears here
Crucially, this AI-generated harmlessness comparison data did not replace human comparison data outright; it was combined with it. The paper reports 135,296 human helpfulness comparisons and 182,831 constitutionally generated harmlessness comparisons feeding one preference model, trained on the union of both, in the authors’ words: “we use human labels for helpfulness, but only AI labels for harmlessness.” A single Bradley-Terry-style preference loss is fit across both sources at once. Writing for the human comparison set, for the AI-generated set, for the reward model, x for a prompt, and , for the winning and losing response in a pair, the objective is
Read the representative guide
Symbol y_l
is a part of this expression. Its role is fixed by the surrounding article and by the operations shown in the formula.
Read this term in its guide →How to interpret it
Read this expression with the definitions, units, and assumptions supplied by the article.
Research cited beside this formula
Published contexts (1)
A symbol can carry a different meaning in another article. Each occurrence keeps its own guide and term definitions.
Equation 5 · Foundation Models
How Constitutional AI Actually Constrains a Model's Behavior
This mathematical expression combines the displayed quantities; its precise role follows from the surrounding article text.
Crucially, this AI-generated harmlessness comparison data did not replace human comparison data outright; it was combined with it. The paper reports 135,296 human helpfulness comparisons and 182,831 constitutionally generated harmlessness comparisons feeding one preference model, trained on the union of both, in the authors’ words: “we use human labels for helpfulness, but only AI labels for harmlessness.” A single Bradley-Terry-style preference loss is fit across both sources at once. Writing for the human comparison set, for the AI-generated set, for the reward model, x for a prompt, and , for the winning and losing response in a pair, the objective is
Equation guide → · Article →