← All parts of this equation

Equation 8 · Part 6 · How Llama's Architecture Actually Works, Generation by Generation

Symbol x_2i

x2i−1′=x2i−1cos⁡(mθi)−x2isin⁡(mθi)x2i′=x2i−1sin⁡(mθi)+x2icos⁡(mθi)\begin{aligned} x'_{2i-1} &= x_{2i-1}\cos(m\theta_i) - x_{2i}\sin(m\theta_i) \\ x'_{2i} &= x_{2i-1}\sin(m\theta_i) + x_{2i}\cos(m\theta_i) \end{aligned}
x2ix_{2i}

What this part means

x2x_2i is one of the signed contributions combined to compute the quantity on the left.

Its job in the formula

x2x_2i is one of the signed contributions combined to compute the quantity on the left.

The passage around this formula

and a vector at sequence position m has its i -th pair rotated by angle mθi\theta_i : x2i−1′=x2i−1cos⁡(mθi)−x2isin⁡(mθi)x2i′=x2i−1sin⁡(mθi)+x2icos⁡(mθi)\begin{aligned} x'_{2i-1} &= x_{2i-1}\cos(m\theta_i) - x_{2i}\sin(m\theta_i) \\ x'_{2i} &= x_{2i-1}\sin(m\theta_i) + x_{2i}\cos(m\theta_i) \end{aligned}. The property that makes this useful rather than merely elegant is that rotating a query at position m and a key at position n by these matrices before taking their dot product leaves a result that depends only on the distance between them:

Read this part in the article →

Learn the underlying idea

A subscript is a label attached below a symbol. It often selects a time step, component, category, or member of a sequence.

Open the illustrated subscripts: which member of a family? guide →

See this notation across published equations →

Sources cited in the article section

These citations provide research context; check each source for the exact claim it supports.