Symbol d_i^(m_2)
d_i^() is part of the quantity the equation computes from the expression on the right.
Read this term in its guide →Published equation contexts
Why this formula appears here
Leask, Nanda and colleagues then tested the assumption directly with two new techniques, and the result undercuts the idea that there is a “right” dictionary size waiting to be found by scaling further. Stitching the dictionaries of differently sized sparse autoencoders together, they show that larger dictionaries recover latents genuinely missing from smaller ones — the smaller dictionary is incomplete. Training a second, “meta” sparse autoencoder on the decoder directions of a first one, they show that what looks like a single, atomic feature in a large dictionary is itself well approximated by a sparse combination of directions from a smaller one — a reported example decomposes an…
Read the representative guide
Symbol j
j appears in the bound of this sum. The bound states where the repeated operation starts, ends, or which values it includes.
Read this term in its guide →Symbol m_1
appears in the bound of this sum. The bound states where the repeated operation starts, ends, or which values it includes.
Read this term in its guide →Symbol g_j
is an input to the expression that computes the quantity on the left.
Read this term in its guide →Symbol d_j^(m_1)
d_j^() is an input to the expression that computes the quantity on the left.
Read this term in its guide →Symbol g
g is an input to the expression that computes the quantity on the left.
Read this term in its guide →Symbol m_2
is an argument of the function-like quantity on the left; its role is set by that function’s stated inputs.
Read this term in its guide →Starting index or lower bound: j=1
This label says where the repeated addition, multiplication, or accumulation starts. Read its value or condition together with the article’s description of the index.
Read this term in its guide →Ending index or upper bound: m_1
This label says where the repeated addition, multiplication, or accumulation stops. It sets the last term or end of the range.
Read this term in its guide →How to interpret it
Its accuracy depends on the assumptions and range of use described in the article. Read it with the definitions, units, and assumptions supplied by the article.
Research cited beside this formula
Published contexts (1)
A symbol can carry a different meaning in another article. Each occurrence keeps its own guide and term definitions.
Equation 5 · AI Research
The Hardest Unsolved Problems in Mechanistic Interpretability
This equation gives an approximation: it relates the quantities while allowing an approximation.
Leask, Nanda and colleagues then tested the assumption directly with two new techniques, and the result undercuts the idea that there is a “right” dictionary size waiting to be found by scaling further. Stitching the dictionaries of differently sized sparse autoencoders together, they show that larger dictionaries recover latents genuinely missing from smaller ones — the smaller dictionary is incomplete. Training a second, “meta” sparse autoencoder on the decoder directions of a first one, they show that what looks like a single, atomic feature in a large dictionary is itself well approximated by a sparse combination of directions from a smaller one — a reported example decomposes an…
Equation guide → · Article →