← Mathematical compendium

Published equation contexts

J(θ)=Ex∼D, y∼πθ(⋅∣x)[r(x,y)]\mathcal{J}(\theta) = \mathbb{E}_{x \sim \mathcal{D},\ y \sim \pi_\theta(\cdot \mid x)}\left[ r(x, y) \right]

Why this formula appears here

The synthesis was to stop prompting for deliberation and start training for it, using the fact that some answers can be checked automatically. Lambert and colleagues named the technique in the open literature as Reinforcement Learning with Verifiable Rewards, applied within an otherwise conventional post-training pipeline [ 22 ] . The objective is unusually simple: J(θ)=Ex∼D, y∼πθ(⋅∣x)[r(x,y)]\mathcal{J}(\theta) = \mathbb{E}_{x \sim \mathcal{D},\ y \sim \pi_\theta(\cdot \mid x)}\left[ r(x, y) \right]. where r is not a learned preference model but a program: a unit test that passes, a numerical answer that matches, a proof that checks. Because r is exact, it cannot be gamed the way a learned reward model can, though it is only available where correctness is mechanically decidable.

Read the full article-specific guide →

Read the representative guide

θ\theta

Symbol θ

θ is an argument of the function-like quantity on the left; its role is set by that function’s stated inputs.

Read this term in its guide →
Ex∼D, y∼πθ(⋅∣x)\mathbb{E}_{x \sim \mathcal{D},\ y \sim \pi_\theta(\cdot \mid x)}

Symbol E_x sim D, y sim pi_θ( × mid x)

ExE_x sim D, y sim pi_θ( × mid x) appears inside an expected value, so its contribution is averaged under the distribution or condition shown by that operator.

Read this term in its guide →
rr

Symbol r

r appears inside an expected value, so its contribution is averaged under the distribution or condition shown by that operator.

Read this term in its guide →
xx

Symbol x

x appears inside an expected value, so its contribution is averaged under the distribution or condition shown by that operator.

Read this term in its guide →
yy

Symbol y

y appears inside an expected value, so its contribution is averaged under the distribution or condition shown by that operator.

Read this term in its guide →

How to interpret it

Read it with the definitions, units, and assumptions supplied by the article.

Research cited beside this formula

Learn the mathematics behind it

Published contexts (1)

A symbol can carry a different meaning in another article. Each occurrence keeps its own guide and term definitions.

J(θ)=Ex∼D, y∼πθ(⋅∣x)[r(x,y)],\mathcal{J}(\theta) = \mathbb{E}_{x \sim \mathcal{D},\ y \sim \pi_\theta(\cdot \mid x)}\left[ r(x, y) \right],

Equation 8 · Foundation Models

From n-Grams to Reasoning Models: A Technical History of the Language Model

This equation states an equality: the expressions on both sides have the same value under the article’s assumptions.

The synthesis was to stop prompting for deliberation and start training for it, using the fact that some answers can be checked automatically. Lambert and colleagues named the technique in the open literature as Reinforcement Learning with Verifiable Rewards, applied within an otherwise conventional post-training pipeline [ 22 ] . The objective is unusually simple: J(θ)=Ex∼D, y∼πθ(⋅∣x)[r(x,y)]\mathcal{J}(\theta) = \mathbb{E}_{x \sim \mathcal{D},\ y \sim \pi_\theta(\cdot \mid x)}\left[ r(x, y) \right]. where r is not a learned preference model but a program: a unit test that passes, a numerical answer that matches, a proof that checks. Because r is exact, it cannot be gamed the way a learned reward model can, though it is only available where correctness is mechanically decidable.

Equation guide → · Article →