← All symbols

Symbol in context

zz

This entry links every published equation using this exact notation. Its meaning may change between equations.

Meanings in context

Used in 36 equations

FFNSwiGLU(x)=(Swish1(xW1)⊙xW3) W2,Swish1(z)=z⋅σ(z).\mathrm{FFN}_{\mathrm{SwiGLU}}(x) = \big(\mathrm{Swish}_1(xW_1) \odot xW_3\big)\,W_2, \qquad \mathrm{Swish}_1(z) = z \cdot \sigma(z).

How Llama's Architecture Actually Works, Generation by Generation · Equation 16

This equation states an equality: the expressions on both sides have the same value under the article’s assumptions.

Equation guide → · This term → · Article →
LSAE(a)=∥a−a^∥22+λ∥z∥1,a^=Wdec z+bdec,z=ReLU ⁣(Wenc(a−bdec)+benc).\mathcal{L}_{\mathrm{SAE}}(a) = \lVert a - \hat a \rVert_2^2 + \lambda \lVert z \rVert_1, \qquad \hat a = W_{\mathrm{dec}}\, z + b_{\mathrm{dec}}, \qquad z = \mathrm{ReLU}\!\left(W_{\mathrm{enc}}(a - b_{\mathrm{dec}}) + b_{\mathrm{enc}}\right).

How Mechanistic Interpretability Research Is Actually Done · Equation 9

This equation states an equality: the expressions on both sides have the same value under the article’s assumptions.

Equation guide → · This term → · Article →
p(y∣x)≈∑z∈top-k(pη(⋅∣x))pη(z∣x) pθ(y∣x,z),p(y \mid x) \approx \sum_{z \in \mathrm{top}\text{-}k\left(p_\eta(\cdot \mid x)\right)} p_\eta(z \mid x) \, p_\theta(y \mid x, z),

From BM25 to Agentic Retrieval: A History of Retrieval-Augmented Generation · Equation 21

This equation gives an approximation: it relates the quantities while allowing an approximation.

Equation guide → · This term → · Article →