Equation 13 · Part 4 · How Mechanistic Interpretability Research Is Actually Done

Symbol a

z = \mathrm{TopK}_k\!\left(W_{\mathrm{enc}}(a - b_{\mathrm{dec}})\right),

a

What this part means

the writing.

Its job in the formula

a is one of the signed contributions combined to compute the quantity on the left.

Full expressionSymbol aArticle meaning

Where the article explains it

Writing a for the activation, z for its sparse code and $\hat a$ for the reconstruction, $\mathcal{L}_{\mathrm{SAE}}(a)$ = $\lVert$ a - $\hat a$ $\rVert_2^2$ + $\lambda$ $\lVert$ z $\rVert_1$ , $\qquad$ $\hat a$ = $W_{\mathrm{dec}}$ \, z + $b_{\mathrm{dec}}$ , $\qquad$ z = $\mathrm{ReLU}$ \!

The passage around this formula

The $\ell_1$ penalty has a known cost: it does not directly control how many dictionary elements fire, only how much their combined magnitude is discouraged, and it systematically shrinks the elements that do fire toward zero, biasing the reconstruction. Two later refinements address this more directly. Gao and colleagues…

Read this part in the article →

Learn the underlying idea

A variable is a named place for a value. Its letter is a local label: x can mean position in one formula and a data point in another.

Open the illustrated variables: a letter stands for a value guide →

See this notation across published equations →

Sources cited in the surrounding passage

These citations provide research context; check each source for the exact claim it supports.