Symbol in context

$D_{\mathrm{KL}}$

This entry links every published equation using this exact notation. Its meaning may change between equations.

Used in 4 equations

D_{\mathrm{KL}}(p \,\|\, q) = \sum_i p_i \log_2(p_i/q_i)

This equation states an equality: the expressions on both sides have the same value under the article’s assumptions.

\max_\theta \; \mathbb{E}_{x,\, y\sim\pi_\theta}\big[r_\phi(x,y)\big] \;-\; \beta \, D_{\mathrm{KL}}\big(\pi_\theta \,\Vert\, \pi_{\mathrm{ref}}\big).

This mathematical expression combines the displayed quantities; its precise role follows from the surrounding article text.

J(\theta) = \mathbb{E}_{x \sim \mathcal{D},\ y \sim \pi_\theta(\cdot \mid x)}\left[r(x,y)\right] - \beta D_{\mathrm{KL}}\left(\pi_\theta(\cdot \mid x) \,\|\, \pi_{\mathrm{ref}}(\cdot \mid x)\right)

This equation states an equality: the expressions on both sides have the same value under the article’s assumptions.

\mathbb{E}\big[r_\theta(x,y)\big] - \beta\,\mathbb{E}\big[D_{\mathrm{KL}}\!\left(\pi^{\mathrm{RL}} \,\|\, \pi^{\mathrm{SFT}}\right)\big] + \gamma\,\mathbb{E}\big[\log \pi^{\mathrm{RL}}(x)\big],

This mathematical expression combines the displayed quantities; its precise role follows from the surrounding article text.