Equation 16 · Part 2 · How Mechanistic Interpretability Research Is Actually Done
subscript
subscript
What this part means
The lower label selects a particular version, component, or indexed member of the quantity. For example, x₀ and xₜ can be values at different positions.
Its job in the formula
A subscript distinguishes a version, component, step, or member of a quantity. It does not automatically mean multiplication.
Full expression→subscript→Article meaning
The passage around this formula
keeping only the k largest pre-activations and zeroing the rest, and used it to train a sixteen-million-latent dictionary on GPT-4 activations over forty billion tokens, reporting metrics that improve consistently as dictionary size grows [ 8 ] . Rajamanoharan and colleagues took a different route to the same problem, keeping a continuous encoder but replacing the fixed zero threshold with a learned per-feature threshold — a feature only activates once its pre-activation clears — and report state-of-the-art reconstruction fidelity at matched sparsity on Gemma 2 activations against both the and top- k alternatives [ 9 ] .
Learn the underlying idea
A subscript is a label attached below a symbol. It often selects a time step, component, category, or member of a sequence.
Open the illustrated subscripts: which member of a family? guide →
Sources cited in the surrounding passage
- [8] Scaling and evaluating sparse autoencoders ↗
- [9] Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders ↗
These citations provide research context; check each source for the exact claim it supports.