Equation 10 · How Mechanistic Interpretability Research Is Actually Done
What does this equation mean?
Read the formula alongside the article passage below. Each part has a deeper page with its role in the equation, the supporting passage and nearby citations.
This mathematical expression combines the displayed quantities; its precise role follows from the surrounding article text. Read the equation part by part below; each part has a contextual explanation and a link to its mathematical background.
Read it piece by piece
subscript
The lower label selects a particular version, component, or indexed member of the quantity. For example, x₀ and xₜ can be values at different positions.
How to interpret it
Read this expression with the definitions, units, and assumptions supplied by the article.
What the article says around this equation
The reconstruction term asks the dictionary to explain the activation; the penalty asks it to explain it using as few active dictionary elements as possible at once. Cunningham and colleagues showed this produces directions substantially more interpretable than neurons or principal components, and — the operational payoff — that the recovered directions support finer-grained causal attribution of specific behaviours than the alternatives available at the time [ 6 ] . Anthropic’s dictionary-learning demonstration on a one-layer model, published the same season, is the paper most responsible for making this the default first move in a new interpretability project rather than one…
Read the full surrounding passage
The reconstruction term asks the dictionary to explain the activation; the penalty asks it to explain it using as few active dictionary elements as possible at once. Cunningham and colleagues showed this produces directions substantially more interpretable than neurons or principal components, and — the operational payoff — that the recovered directions support finer-grained causal attribution of specific behaviours than the alternatives available at the time [ 6 ] . Anthropic’s dictionary-learning demonstration on a one-layer model, published the same season, is the paper most responsible for making this the default first move in a new interpretability project rather than one technique among several [ 7 ] .
Sources cited in the surrounding passage
- [6] Sparse Autoencoders Find Highly Interpretable Features in Language Models ↗
- [7] Towards Monosemanticity: Decomposing Language Models With Dictionary Learning ↗
These citations give research context. Read each source to check which claims it supports.
Return to How Mechanistic Interpretability Research Is Actually Done