$$\mathcal{L}_{\mathrm{SAE}}(a) = \lVert a - \hat a \rVert_2^2 + \lambda \lVert z \rVert_1, \qquad \hat a = W_{\mathrm{dec}}\, z + b_{\mathrm{dec}}, \qquad z = \mathrm{ReLU}\!\left(W_{\mathrm{enc}}(a - b_{\mathrm{dec}}) + b_{\mathrm{enc}}\right)$$

Equation 9 · AI Research

How Mechanistic Interpretability Research Is Actually Done

This equation states an equality: the expressions on both sides have the same value under the article’s assumptions.

Sparse dictionary learning is the field’s answer, and it is a second and different act of extraction rather than a departure from the first: a sparse autoencoder is trained on the very same activation vectors the probe was reading, learning an overcomplete basis in which each vector is reconstructed as a sparse combination of dictionary elements. Writing a for the activation, z for its sparse code and $\hat a$ for the reconstruction, $\mathcal{L}_{\mathrm{SAE}}(a) = \lVert a - \hat a \rVert_2^2 + \lambda \lVert z \rVert_1, \qquad \hat a = W_{\mathrm{dec}}\, z + b_{\mathrm{dec}}, \qquad z = \mathrm{ReLU}\!\left(W_{\mathrm{enc}}(a - b_{\mathrm{dec}}) + b_{\mathrm{enc}}\right)$. The reconstruction term asks the dictionary to explain the activation; the $\ell_1$ penalty asks it to explain it using as few active dictionary elements as possible at once. Cunningham and colleagues showed this produces directions…

Meanings in this article

• $a$: the writing.

Equation guide → · Article →