← Mathematical compendium

Published equation contexts

θi=base−2(i−1)/d,i=1,…,d/2\theta_i = \text{base}^{-2(i-1)/d}, \qquad i = 1, \dots, d/2

Why this formula appears here

Split each query or key vector of dimension d into d/2 two-dimensional pairs. Pair i is assigned a fixed rotation frequency θi=base−2(i−1)/d,i=1,…,d/2\theta_i = \text{base}^{-2(i-1)/d}, \qquad i = 1, \dots, d/2. and a vector at sequence position m has its i -th pair rotated by angle mθi\theta_i :

Read the full article-specific guide →

Read the representative guide

θi\theta_i

Symbol theta_i

thetaia_i is part of the quantity the equation computes from the expression on the right.

Read this term in its guide →

How to interpret it

Read it with the definitions, units, and assumptions supplied by the article.

Research cited beside this formula

Learn the mathematics behind it

Published contexts (1)

A symbol can carry a different meaning in another article. Each occurrence keeps its own guide and term definitions.

θi=base−2(i−1)/d,i=1,…,d/2,\theta_i = \text{base}^{-2(i-1)/d}, \qquad i = 1, \dots, d/2,

Equation 4 · Open Models

How Llama's Architecture Actually Works, Generation by Generation

This equation states an equality: the expressions on both sides have the same value under the article’s assumptions.

Split each query or key vector of dimension d into d/2 two-dimensional pairs. Pair i is assigned a fixed rotation frequency θi=base−2(i−1)/d,i=1,…,d/2\theta_i = \text{base}^{-2(i-1)/d}, \qquad i = 1, \dots, d/2. and a vector at sequence position m has its i -th pair rotated by angle mθi\theta_i :

Equation guide → · Article →