← All parts of this equation

Equation 4 · Part 1 · How Llama's Architecture Actually Works, Generation by Generation

Symbol theta_i

θi=base−2(i−1)/d,i=1,…,d/2,\theta_i = \text{base}^{-2(i-1)/d}, \qquad i = 1, \dots, d/2,
θi\theta_i

What this part means

thetaia_i is part of the quantity the equation computes from the expression on the right.

Its job in the formula

thetaia_i is part of the quantity the equation computes from the expression on the right.

The passage around this formula

Split each query or key vector of dimension d into d/2 two-dimensional pairs. Pair i is assigned a fixed rotation frequency θi=base−2(i−1)/d,i=1,…,d/2\theta_i = \text{base}^{-2(i-1)/d}, \qquad i = 1, \dots, d/2. and a vector at sequence position m has its i -th pair rotated by angle mθi\theta_i :

Read this part in the article →

Learn the underlying idea

A subscript is a label attached below a symbol. It often selects a time step, component, category, or member of a sequence.

Open the illustrated subscripts: which member of a family? guide →

See this notation across published equations →

Sources cited in the article section

These citations provide research context; check each source for the exact claim it supports.