← All parts of this equation

Equation 9 · Part 3 · Probing, Sparse Autoencoders, Patching, and Steering: The Main Interpretability Methods, Compared

Symbol f

x^=Wdf(x)+bd,f(x)=JumpReLUθ(Wex+be),L(x)=∥x−x^∥22+λ∥f(x)∥0.\hat x = W_d f(x) + b_d, \qquad f(x) = \mathrm{JumpReLU}_\theta\big(W_e x + b_e\big), \qquad \mathcal L(x) = \lVert x - \hat x \rVert_2^2 + \lambda \lVert f(x) \rVert_0.
ff

What this part means

f is one of the signed contributions combined to compute the quantity on the left.

Its job in the formula

f is one of the signed contributions combined to compute the quantity on the left.

The passage around this formula

The formal objective has evolved since the earliest versions, and the direction of that evolution is itself informative. A dictionary decoder reconstructs the activation x from a sparse code f(x) : x^=Wdf(x)+bd,f(x)=JumpReLUθ(Wex+be),L(x)=∥x−x^∥22+λ∥f(x)∥0\hat x = W_d f(x) + b_d, \qquad f(x) = \mathrm{JumpReLU}_\theta\big(W_e x + b_e\big), \qquad \mathcal L(x) = \lVert x - \hat x \rVert_2^2 + \lambda \lVert f(x) \rVert_0. Earlier versions of this objective penalised the code’s ℓ1\ell_1 norm as a differentiable stand-in for sparsity, but an ℓ1\ell_1 penalty also shrinks the magnitude of every active feature, distorting reconstruction in a way that has nothing to do with how many features are…

Read this part in the article →

Learn the underlying idea

A function assigns an output to each allowed input. The expression f(x) means “apply f to x”.

Open the illustrated functions: inputs become outputs guide →

See this notation across published equations →

Sources cited in the surrounding passage

These citations provide research context; check each source for the exact claim it supports.