Equation 25 · Part 5 · The Main Technical Approaches to AI Alignment, Compared
=
=
What this part means
The expressions on both sides represent the same quantity under the stated assumptions.
Its job in the formula
The equals sign connects the complete expression on the left with the complete expression on the right. Both sides must have compatible units.
Full expression→=→Article meaning
The passage around this formula
normalizing each sampled response’s reward against the mean and standard deviation of a group of G responses to the same prompt rather than against a separately trained critic network [ 13 ] . Lambert and colleagues, building the fully open Tulu 3 post-training recipe, named the general approach explicitly, describing it as “a novel method we call Reinforcement Learning with Verifiable Rewards,” and used it alongside supervised fine-tuning and preference optimization rather than as a wholesale replacement for either [ 15 ] . The clearest large-scale demonstration is DeepSeek-R1: Guo and colleagues report training a model with reinforcement learning alone against a purely rule-based reward —…
Learn the underlying idea
An equals sign says that the expression on its left and the expression on its right have the same value under the stated definitions and assumptions.
Open the illustrated equality: what the equals sign claims guide →
Sources cited in the surrounding passage
- [13] DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models ↗
- [15] Tulu 3: Pushing Frontiers in Open Language Model Post-Training ↗
- [14] DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning ↗
These citations provide research context; check each source for the exact claim it supports.