← All symbols

Symbol in context

GG

This entry links every published equation using this exact notation. Its meaning may change between equations.

Meanings in context

Used in 41 equations

G∈{1,2,…,H},G=H⇒multi-head,G=1⇒multi-query,1<G<H⇒grouped-query.G \in \{1, 2, \dots, H\}, \qquad G = H \Rightarrow \text{multi-head}, \quad G = 1 \Rightarrow \text{multi-query}, \quad 1 < G < H \Rightarrow \text{grouped-query}.

How Llama's Architecture Actually Works, Generation by Generation · Equation 15

This equation states a bound: one expression must stay on the indicated side of the other under the article’s assumptions.

Equation guide → · This term → · Article →
y(x)=∑i∈TopK(G(x))G(x)i⋅Ei(x),Ctok≈kE⋅Ctokdense(Ntotal)y(x) = \sum_{i \in \mathrm{TopK}(G(x))} G(x)_i \cdot E_i(x), \qquad C_{\text{tok}} \approx \frac{k}{E} \cdot C_{\text{tok}}^{\text{dense}}(N_{\text{total}})

Shrink It, Train It Small, or Search for It: The Main Strategies for Small Models, Compared · Equation 29

This equation gives an approximation: it relates the quantities while allowing an approximation.

Equation guide → · This term → · Article →