Equation 10 · Part 1 · What Interpretability Actually Costs to Do at Scale
Symbol d
What this part means
d is a part of this expression. Its role is fixed by the surrounding article and by the operations shown in the formula.
Its job in the formula
d is a part of this expression. Its role is fixed by the surrounding article and by the operations shown in the formula.
Full expression→Symbol d→Article meaning
The passage around this formula
Work out what that architecture actually spends compute on per token, because the two halves of it behave differently. The encoding step needs a score for every one of the n candidate latents before it can select the top k , so it is an unavoidably dense matrix multiply: roughly 2dn floating-point operations. The decoding step only touches the k latents that survived, so it is sparse: roughly 2dk operations. Summed and multiplied across T training tokens, a first-order compute model for training the dictionary is
Learn the underlying idea
A variable is a named place for a value. Its letter is a local label: x can mean position in one formula and a data point in another.
Open the illustrated variables: a letter stands for a value guide →
See this notation across published equations →
Sources cited in the article section
These citations provide research context; check each source for the exact claim it supports.