← Back to article

Equation 1 · The Hardest Unsolved Problems in Mechanistic Interpretability

What does this equation mean?

di(m2)d_i^{(m_2)}

Read the formula alongside the article passage below. Each part has a deeper page with its role in the equation, the supporting passage and nearby citations.

This mathematical expression combines the displayed quantities; its precise role follows from the surrounding article text. Read the equation part by part below; each part has a contextual explanation and a link to its mathematical background.

Read it piece by piece

di(m2)d_i^{(m_2)}

Symbol d_i^(m_2)

d_i^(m2m_2) is a part of this expression. Its role is fixed by the surrounding article and by the operations shown in the formula.

Understand this part →

subscript

subscript

The lower label selects a particular version, component, or indexed member of the quantity. For example, x₀ and xₜ can be values at different positions.

Understand this part →

superscript

superscript

A raised number can be a power. When it is a label or bound, it selects a case or the upper limit of a sum; the formula’s structure distinguishes these uses.

Understand this part →

See an illustrated explanation →

How to interpret it

Read this expression with the definitions, units, and assumptions supplied by the article.

What the article says around this equation

Leask, Nanda and colleagues then tested the assumption directly with two new techniques, and the result undercuts the idea that there is a “right” dictionary size waiting to be found by scaling further. Stitching the dictionaries of differently sized sparse autoencoders together, they show that larger dictionaries recover latents genuinely missing from smaller ones — the smaller dictionary is incomplete. Training a second, “meta” sparse autoencoder on the decoder directions of a first one, they show that what looks like a single, atomic feature in a large dictionary is itself well approximated by a sparse combination of directions from a smaller one — a reported example decomposes an…
Read the full surrounding passage
Leask, Nanda and colleagues then tested the assumption directly with two new techniques, and the result undercuts the idea that there is a “right” dictionary size waiting to be found by scaling further. Stitching the dictionaries of differently sized sparse autoencoders together, they show that larger dictionaries recover latents genuinely missing from smaller ones — the smaller dictionary is incomplete. Training a second, “meta” sparse autoencoder on the decoder directions of a first one, they show that what looks like a single, atomic feature in a large dictionary is itself well approximated by a sparse combination of directions from a smaller one — a reported example decomposes an “Einstein” feature into components resembling “scientist,” “Germany,” and “famous person.” Writing di(m2)d_i^{(m_2)} for one feature direction recovered by a dictionary of size m2m_2 and dj(m1)d_j^{(m_1)} for the directions recovered by a smaller dictionary of size m1m_1 , their finding is that

Read the equation in its article →

Sources cited in the article section

These citations give research context. Read each source to check which claims it supports.

Return to The Hardest Unsolved Problems in Mechanistic Interpretability

Browse the mathematical compendium →