subscript

The passage around this formula

The synthesis was to stop prompting for deliberation and start training for it, using the fact that some answers can be checked automatically. Lambert and colleagues named the technique in the open literature as Reinforcement Learning with Verifiable Rewards, applied within an otherwise conventional post-training pipeline [ 22 ] . The objective is unusually simple: $\mathcal{J}(\theta) = \mathbb{E}_{x \sim \mathcal{D},\ y \sim \pi_\theta(\cdot \mid x)}\left[ r(x, y) \right]$. where r is not a learned preference model but a program: a unit test that passes, a numerical answer that matches, a proof that checks. Because r is exact, it cannot be gamed the way a learned reward model can, though it is only available where correctness is mechanically decidable.

Read this part in the article →