← All symbols

Symbol in context

xx

This entry links every published equation using this exact notation. Its meaning may change between equations.

Meanings in context

Used in 118 equations

max⁡θ  Ex, y∼πθ[rϕ(x,y)]  −  β DKL(πθ ∥ πref).\max_\theta \; \mathbb{E}_{x,\, y\sim\pi_\theta}\big[r_\phi(x,y)\big] \;-\; \beta \, D_{\mathrm{KL}}\big(\pi_\theta \,\Vert\, \pi_{\mathrm{ref}}\big).

The Main Technical Approaches to AI Alignment, Compared · Equation 2

This mathematical expression combines the displayed quantities; its precise role follows from the surrounding article text.

Equation guide → · This term → · Article →
x2i−1′=x2i−1cos⁡(mθi)−x2isin⁡(mθi)x2i′=x2i−1sin⁡(mθi)+x2icos⁡(mθi)\begin{aligned} x'_{2i-1} &= x_{2i-1}\cos(m\theta_i) - x_{2i}\sin(m\theta_i) \\ x'_{2i} &= x_{2i-1}\sin(m\theta_i) + x_{2i}\cos(m\theta_i) \end{aligned}

How Llama's Architecture Actually Works, Generation by Generation · Equation 8

This equation states an equality: the expressions on both sides have the same value under the article’s assumptions.

Equation guide → · This term → · Article →
FFNSwiGLU(x)=(Swish1(xW1)⊙xW3) W2,Swish1(z)=z⋅σ(z).\mathrm{FFN}_{\mathrm{SwiGLU}}(x) = \big(\mathrm{Swish}_1(xW_1) \odot xW_3\big)\,W_2, \qquad \mathrm{Swish}_1(z) = z \cdot \sigma(z).

How Llama's Architecture Actually Works, Generation by Generation · Equation 16

This equation states an equality: the expressions on both sides have the same value under the article’s assumptions.

Equation guide → · This term → · Article →
RMSNorm(x)j=xjRMS(x) gj,RMS(x)=1d∑k=1dxk2+ϵ.\mathrm{RMSNorm}(x)_j = \frac{x_j}{\mathrm{RMS}(x)}\, g_j, \qquad \mathrm{RMS}(x) = \sqrt{\frac{1}{d}\sum_{k=1}^{d} x_k^2 + \epsilon}.

How Llama's Architecture Actually Works, Generation by Generation · Equation 19

This equation states an equality: the expressions on both sides have the same value under the article’s assumptions.

Equation guide → · This term → · Article →
y(x)=∑i∈TopK(G(x))G(x)i⋅Ei(x),Ctok≈kE⋅Ctokdense(Ntotal)y(x) = \sum_{i \in \mathrm{TopK}(G(x))} G(x)_i \cdot E_i(x), \qquad C_{\text{tok}} \approx \frac{k}{E} \cdot C_{\text{tok}}^{\text{dense}}(N_{\text{total}})

Shrink It, Train It Small, or Search for It: The Main Strategies for Small Models, Compared · Equation 29

This equation gives an approximation: it relates the quantities while allowing an approximation.

Equation guide → · This term → · Article →