← All parts of this equation

Equation 4 · Part 5 · How Llama's Architecture Actually Works, Generation by Generation

subtraction

θi=base−2(i−1)/d,i=1,…,d/2,\theta_i = \text{base}^{-2(i-1)/d}, \qquad i = 1, \dots, d/2,
subtraction

What this part means

Subtract the following term or group from the preceding one. A leading minus marks a negative quantity.

Its job in the formula

Subtract the following term or group from the preceding one. A leading minus marks a negative quantity.

The passage around this formula

Split each query or key vector of dimension d into d/2 two-dimensional pairs. Pair i is assigned a fixed rotation frequency θi=base−2(i−1)/d,i=1,…,d/2\theta_i = \text{base}^{-2(i-1)/d}, \qquad i = 1, \dots, d/2. and a vector at sequence position m has its i -th pair rotated by angle mθi\theta_i :

Read this part in the article →

Learn the underlying idea

Addition combines quantities; subtraction measures the signed difference between them. Parentheses show what is combined before the rest of the expression is evaluated.

Open the illustrated addition and subtraction in an equation guide →

Sources cited in the article section

These citations provide research context; check each source for the exact claim it supports.