← All parts of this equation

Equation 4 · Part 2 · How Llama's Architecture Actually Works, Generation by Generation

Symbol i

θi=base−2(i−1)/d,i=1,…,d/2,\theta_i = \text{base}^{-2(i-1)/d}, \qquad i = 1, \dots, d/2,
ii

What this part means

i is part of the quantity the equation computes from the expression on the right.

Its job in the formula

i is part of the quantity the equation computes from the expression on the right.

The passage around this formula

Split each query or key vector of dimension d into d/2 two-dimensional pairs. Pair i is assigned a fixed rotation frequency θi=base−2(i−1)/d,i=1,…,d/2\theta_i = \text{base}^{-2(i-1)/d}, \qquad i = 1, \dots, d/2. and a vector at sequence position m has its i -th pair rotated by angle mθi\theta_i :

Read this part in the article →

Learn the underlying idea

A variable is a named place for a value. Its letter is a local label: x can mean position in one formula and a data point in another.

Open the illustrated variables: a letter stands for a value guide →

See this notation across published equations →

Sources cited in the article section

These citations provide research context; check each source for the exact claim it supports.