← Mathematical compendium

Published equation contexts

x2i−1′=x2i−1cos⁡(mθi)−x2isin⁡(mθi)x2i′=x2i−1sin⁡(mθi)+x2icos⁡(mθi)\begin{aligned} x'_{2i-1} &= x_{2i-1}\cos(m\theta_i) - x_{2i}\sin(m\theta_i) \\ x'_{2i} &= x_{2i-1}\sin(m\theta_i) + x_{2i}\cos(m\theta_i) \end{aligned}

Why this formula appears here

and a vector at sequence position m has its i -th pair rotated by angle mθi\theta_i : x2i−1′=x2i−1cos⁡(mθi)−x2isin⁡(mθi)x2i′=x2i−1sin⁡(mθi)+x2icos⁡(mθi)\begin{aligned} x'_{2i-1} &= x_{2i-1}\cos(m\theta_i) - x_{2i}\sin(m\theta_i) \\ x'_{2i} &= x_{2i-1}\sin(m\theta_i) + x_{2i}\cos(m\theta_i) \end{aligned}. The property that makes this useful rather than merely elegant is that rotating a query at position m and a key at position n by these matrices before taking their dot product leaves a result that depends only on the distance between them:

Read the full article-specific guide →

Read the representative guide

x2i−1x_{2i-1}

Symbol x_2i-1

x2x_2i-1 is one of the signed contributions combined to compute the quantity on the left.

Read this term in its guide →
θi\theta_i

Symbol theta_i

thetaia_i is one of the signed contributions combined to compute the quantity on the left.

Read this term in its guide →

How to interpret it

Read it with the definitions, units, and assumptions supplied by the article.

Research cited beside this formula

Learn the mathematics behind it

Published contexts (1)

A symbol can carry a different meaning in another article. Each occurrence keeps its own guide and term definitions.

x2i−1′=x2i−1cos⁡(mθi)−x2isin⁡(mθi)x2i′=x2i−1sin⁡(mθi)+x2icos⁡(mθi)\begin{aligned} x'_{2i-1} &= x_{2i-1}\cos(m\theta_i) - x_{2i}\sin(m\theta_i) \\ x'_{2i} &= x_{2i-1}\sin(m\theta_i) + x_{2i}\cos(m\theta_i) \end{aligned}

Equation 8 · Open Models

How Llama's Architecture Actually Works, Generation by Generation

This equation states an equality: the expressions on both sides have the same value under the article’s assumptions.

and a vector at sequence position m has its i -th pair rotated by angle mθi\theta_i : x2i−1′=x2i−1cos⁡(mθi)−x2isin⁡(mθi)x2i′=x2i−1sin⁡(mθi)+x2icos⁡(mθi)\begin{aligned} x'_{2i-1} &= x_{2i-1}\cos(m\theta_i) - x_{2i}\sin(m\theta_i) \\ x'_{2i} &= x_{2i-1}\sin(m\theta_i) + x_{2i}\cos(m\theta_i) \end{aligned}. The property that makes this useful rather than merely elegant is that rotating a query at position m and a key at position n by these matrices before taking their dot product leaves a result that depends only on the distance between them:

Meanings in this article

  • ii: the pair.
  • mm: a position index.
Equation guide → · Article →