← All parts of this equation

Equation 11 · Part 8 · How Llama's Architecture Actually Works, Generation by Generation

subscript

⟨RΘ,m q,  RΘ,n k⟩  =  ⟨RΘ, m−n q,  k⟩.\langle R_{\Theta,m}\,q,\; R_{\Theta,n}\,k\rangle \;=\; \langle R_{\Theta,\,m-n}\,q,\; k\rangle.
subscript

What this part means

The lower label selects a particular version, component, or indexed member of the quantity. For example, x₀ and xₜ can be values at different positions.

Its job in the formula

A subscript distinguishes a version, component, step, or member of a quantity. It does not automatically mean multiplication.

The passage around this formula

The property that makes this useful rather than merely elegant is that rotating a query at position m and a key at position n by these matrices before taking their dot product leaves a result that depends only on the distance between them: ⟨RΘ,m q,  RΘ,n k⟩  =  ⟨RΘ, m−n q,  k⟩\langle R_{\Theta,m}\,q,\; R_{\Theta,n}\,k\rangle \;=\; \langle R_{\Theta,\,m-n}\,q,\; k\rangle. Relative position falls out of the mechanism instead of being engineered into it separately, and — as the RoFormer paper’s own experiments emphasise — the same rotation formula applies at any sequence length the rotation frequencies were designed for, which is what makes RoPE the natural place to intervene when a later generation wants a longer context window [ 5 ] .

Read this part in the article →

Learn the underlying idea

A subscript is a label attached below a symbol. It often selects a time step, component, category, or member of a sequence.

Open the illustrated subscripts: which member of a family? guide →

Sources cited in the surrounding passage

These citations provide research context; check each source for the exact claim it supports.