Equation 14 · Part 8 · What Interpretability Actually Costs to Do at Scale
subscript
subscript
What this part means
The lower label selects a particular version, component, or indexed member of the quantity. For example, x₀ and xₜ can be values at different positions.
Its job in the formula
A subscript distinguishes a version, component, step, or member of a quantity. It does not automatically mean multiplication.
Full expression→subscript→Article meaning
The passage around this formula
Work out what that architecture actually spends compute on per token, because the two halves of it behave differently. The encoding step needs a score for every one of the n candidate latents before it can select the top k , so it is an unavoidably dense matrix multiply: roughly 2dn floating-point operations. The decoding step only touches the k latents that survived, so it is sparse: roughly 2dk operations. Summed and multiplied across T training tokens, a first-order compute model for training the dictionary is . Because published TopK configurations keep k in the tens to low hundreds while n runs into the millions, n k and the encoding term dominates almost…
Learn the underlying idea
A subscript is a label attached below a symbol. It often selects a time step, component, category, or member of a sequence.
Open the illustrated subscripts: which member of a family? guide →
Sources cited in the surrounding passage
These citations provide research context; check each source for the exact claim it supports.