Equation 1 · What a Circuit Explains: The State and Limits of Mechanistic Interpretability
What does this equation mean?
Read the formula alongside the article passage below. Each part has a deeper page with its role in the equation, the supporting passage and nearby citations.
the writing. Read the equation part by part below; each part has a contextual explanation and a link to its mathematical background.
Read it piece by piece
How to interpret it
Read this expression with the definitions, units, and assumptions supplied by the article.
What the article says around this equation
Now the careful part. Consider what the training objective actually asks for. Writing x for an activation vector, f(x) for the sparse code and for the reconstruction, the objective has the form
Sources cited in the article section
- [7] Sparse Autoencoders Find Highly Interpretable Features in Language Models ↗
- [8] Scaling and evaluating sparse autoencoders ↗
- [9] Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet ↗
- [10] Automated Interpretability Metrics Do Not Distinguish Trained and Random Transformers ↗
- [11] Are Sparse Autoencoders Useful? A Case Study in Sparse Probing ↗
These citations give research context. Read each source to check which claims it supports.
Return to What a Circuit Explains: The State and Limits of Mechanistic Interpretability