Equation 5 · Part 9 · The Hardest Unsolved Problems in Mechanistic Interpretability
≈
≈
What this part means
Approximately equal to; the equality is not exact.
Its job in the formula
Approximately equal to; the equality is not exact.
Full expression→≈→Article meaning
The passage around this formula
Leask, Nanda and colleagues then tested the assumption directly with two new techniques, and the result undercuts the idea that there is a “right” dictionary size waiting to be found by scaling further. Stitching the dictionaries of differently sized sparse autoencoders together, they show that larger dictionaries recover latents genuinely missing from smaller ones — the smaller dictionary is incomplete. Training a second, “meta” sparse autoencoder on the decoder directions of a first one, they show that what looks like a single, atomic feature in a large dictionary is itself well approximated by a sparse combination of directions from a smaller one — a reported example decomposes an…
Learn the underlying idea
A variable is a named place for a value. Its letter is a local label: x can mean position in one formula and a data point in another.
Open the illustrated variables: a letter stands for a value guide →
Sources cited in the surrounding passage
These citations provide research context; check each source for the exact claim it supports.