Probing, Sparse Autoencoders, Patching, and Steering: The Main Interpretability Methods, Compared

Four questions, not a ladder

Ask a room of practitioners which interpretability method is “best” and the question has already gone wrong. Probing classifiers, sparse autoencoders, activation patching, and steering vectors are not four competitors converging on one answer. They are four instruments built to answer four different questions about a trained network, and a result from one does not confirm, refute, or rank a result from another. Treating them as rungs on a single ladder — probing as the beginner’s tool, steering as the sophisticated payoff — is a category error that this article is built to dismantle.

The four questions, stated plainly, are: is a property linearly recoverable from an activation at all (probing)? Does a layer’s activity decompose into a small dictionary of sparsely active atoms (sparse autoencoders)? Is a specific component causally necessary or sufficient for a specific output (activation patching and causal tracing)? Can behaviour be moved, predictably and by hand, along a direction found in activation space (representation engineering and steering vectors)? Each question has its own mathematical objective, its own originating literature, and its own documented failure modes, reported by the same researchers who built the methods.

This matters because the four are frequently cited interchangeably in secondary coverage — “interpretability shows that the model represents X” — when the underlying claim differs enormously depending on which of the four was actually run. A probing result licenses a much weaker conclusion than a patching result; a steering result licenses a different kind of conclusion again, one about control rather than about explanation. The rest of this article works through each method in turn: what it is built on, what a positive result actually certifies, and what its own literature says it does not certify.

Probing classifiers: is the property linearly there

The oldest and cheapest of the four methods asks the simplest question. Train a small classifier — usually linear — to predict some property of interest from a network’s internal activation at a chosen layer, holding the network itself frozen. If the classifier can be trained to high accuracy, the property is said to be “decodable,” or informally, “present,” at that layer. Alain and Bengio introduced the technique under the name of linear classifier probes, applying it to vision networks and reporting that the linear separability of intermediate features rose monotonically with depth, which they read as evidence that the network was progressively building more useful representations as information passed through its layers ^[1]. The idea was adopted quickly in language processing, where Conneau and colleagues built a battery of ten probing tasks — sentence length, tree depth, word content, and similar syntactic and surface properties — and used them to characterise what different sentence-encoding schemes actually retained ^[2].

Formally, a probe is a function fitted independently of the network under study:

where $h_\ell(x)$ is the frozen activation the network produces at layer $\ell$ for input $x$, and $w, b$ are the probe’s own parameters, trained on labelled examples the network never saw during its own training. Nothing in that objective touches the network’s weights or its downstream computation. A probe reports only whether some linear function of this one activation predicts the label well.

That gap between “predictable from” and “used by” is the method’s central and openly documented weakness. Hewitt and Liang showed that a sufficiently expressive probe can achieve high accuracy predicting properties from representations that plausibly do not encode them in any meaningful sense, because the probe itself has the capacity to memorise idiosyncratic patterns in the training data. Their fix was the control task: construct a version of the labelling scheme that associates each input type with an output at random, so it can only be learned by the probe’s own memorisation capacity, never by any real linguistic signal in the representation. A well-behaved probe should then show high accuracy on the real task and low accuracy on the matched control, and the paper defines the resulting quantity as selectivity:

A probe with low selectivity is not measuring the network; it is measuring itself ^[3]. Belinkov’s survey of the whole probing-classifier programme, published in Computational Linguistics, restates the deeper problem this points to: probing is correlational by construction. A property can be decodable from an activation and still play no causal role in what the network actually outputs, because the network’s downstream layers may never read that component of the representation, or may read it in combination with enough other information that the isolated correlation is misleading. The survey’s own framing is that probing classifiers have genuine promises, real and well-documented shortcomings — chiefly the presence-versus-use gap and the risk of probe overcapacity — and a set of methodological advances, control tasks among them, that partially but not fully close those gaps ^[4].

What a probing result actually licenses, stated precisely: a positive result licenses the claim that the tested property is linearly recoverable at the tested layer, given the tested probe architecture and training procedure, above what a matched control condition would recover. It does not license a claim about whether the network’s own computation depends on that property, still less a claim about where in the network the property is put to use.

Sparse autoencoders: what are the atoms of the code

The second method starts from a specific empirical puzzle that probing cannot resolve: individual neurons in trained networks routinely respond to collections of unrelated inputs with no shared human description, a phenomenon usually called polysemanticity. The leading explanation is superposition. Elhage and colleagues showed, in models simple enough to analyse completely, that a network with fewer dimensions than it has potentially useful features can still represent all of them by assigning features to non-orthogonal directions, accepting a controlled amount of mutual interference in exchange for representing far more concepts than it has room for if each required its own dedicated dimension. Their toy models demonstrated distinct regimes: at low feature sparsity, the network prefers dedicated, near-orthogonal directions; as sparsity rises, it shifts into superposed, overlapping encodings ^[5]. If superposition is why individual neurons are polysemantic, the natural response is to stop treating neurons as the unit of analysis and instead try to recover the individual, less overlapping directions directly.

Sparse dictionary learning does this by training an overcomplete autoencoder on a layer’s activations, with a sparsity constraint pushing most of its hidden units to be inactive on any given input; the active units are then read as candidate features. Bricken and colleagues demonstrated the approach directly on a language model’s residual-stream activations, training sparse autoencoders whose recovered dictionary elements were, by their account, considerably more specific and more interpretable than either raw neurons or principal components, responding to particular and often abstract concepts and producing predictable changes in generation when a chosen element was clamped on or off ^[6].

The formal objective has evolved since the earliest versions, and the direction of that evolution is itself informative. A dictionary decoder reconstructs the activation $x$ from a sparse code $f(x)$:

Earlier versions of this objective penalised the code’s $\ell_1$ norm as a differentiable stand-in for sparsity, but an $\ell_1$ penalty also shrinks the magnitude of every active feature, distorting reconstruction in a way that has nothing to do with how many features are active. Rajamanoharan and colleagues introduced JumpReLU, a thresholded activation function with a learned per-feature cutoff $\theta$ trained through a straight-through gradient estimator, which lets the objective penalise the true count of active features, $\lVert f(x) \rVert_0$, directly rather than through the $\ell_1$ proxy, and reported state-of-the-art reconstruction fidelity at matched sparsity against both the earlier $\ell_1$ formulation and a competing gated variant ^[7]. That the field kept revising the sparsity penalty is itself evidence of how much the objective’s exact shape affects what gets recovered.

The decisive limitation is not about engineering the penalty correctly; it is about what the objective, however penalised, actually certifies. Every term in that loss function refers only to the activation vector $x$ and its reconstruction. No term refers to what the network’s own downstream layers do with that activation. Low reconstruction error at high sparsity is evidence that the activation distribution happens to be sparsely decomposable in the basis the autoencoder found. It is not evidence that this is the basis the network itself treats as its working units. Chanin and colleagues gave this a sharp, concrete demonstration under the name feature absorption: when the true underlying features form a hierarchy — a general property and a more specific one that implies it — the sparsity objective can cause the general feature to stop firing on cases where it logically should, because those cases have been “absorbed” into the more specific child feature instead. The result is a dictionary that looks clean and monosemantic on inspection while silently failing to fire where a human reading of the concept would expect it to, precisely because sparsity, not fidelity to the network’s own computation, is what the objective rewards ^[8]. A sparse autoencoder answers “what is a low-dimensional, sparse basis that reconstructs this activation,” not “what are this network’s own computational units,” and the two questions can have different correct answers.

Activation patching and causal tracing: what does this component actually carry

The third method exists because the first two are correlational. A property can be decodable, and a dictionary element can reconstruct cleanly, without either one being a component the network’s own forward pass actually relies on. The remedy is to intervene rather than observe: change a specific piece of the computation and measure whether the output changes in the way a causal account predicts.

The theoretical grounding for this comes from causal mediation analysis, a framework originally developed in statistics for decomposing a treatment’s total effect on an outcome into the portion that flows through a specified mediator and the portion that does not. Vig and colleagues applied it directly to transformer language models, examining gender bias by testing which individual neurons and attention heads mediated a biased prediction, and reporting that the effect concentrated in a comparatively small number of components at specific layers rather than being diffused evenly across the network ^[9]. That paper supplied the theoretical apparatus; the method that made it tractable at scale for factual computation was causal tracing, introduced by Meng and colleagues. Their procedure runs a model once on a clean prompt and once on a corrupted version of the same prompt, then restores individual hidden states from the clean run into the corrupted run one at a time, measuring how much of the clean output is recovered by restoring each one. They reported that factual recall concentrated causally in a specific set of mid-layer feed-forward modules at the final token of the subject, a localisation precise enough that they then used it to perform a targeted, rank-one edit to a specific stored fact ^[10].

The general operation, activation patching, is usually reported as a normalised score rather than a raw difference, isolating exactly how much of the gap between clean and corrupted behaviour a given component set closes:

where $m_{\text{clean}}$ and $m_{\text{corrupt}}$ are a chosen behavioural metric measured on the clean and corrupted runs, and $m_{\text{patch}}(\mathcal C)$ is that same metric after the activations of component set $\mathcal C$ are copied from one run into the other. A score near one means patching $\mathcal C$ alone restores nearly all of the clean behaviour; a score near zero means it restores almost none.

Heimersheim and Nanda’s methodological account of this technique, written from direct practical experience running it, is the clearest documented source on how easily the result changes under choices that are rarely reported in full. Patching in one direction — copying clean activations into a corrupted run, “denoising” — tests whether $\mathcal C$ is sufficient to restore the behaviour; patching in the other direction — copying corrupted activations into a clean run, “noising” — tests whether $\mathcal C$ is necessary to sustain it, and the two are not mirror images of the same fact about the network. The choice of corruption itself, whether zero-ablation, Gaussian noise, or a resampled alternative prompt, changes which components appear causally load-bearing, and the choice of behavioural metric — a logit difference, a probability, a KL divergence — can shift the ranked importance of components even when every other choice is held fixed. Their central methodological warning is that a patching result is a claim about what happens when this specific substitution is made, under this specific metric and this specific corruption; it is not automatically a claim about how the component functions in general, and readers who are given a localisation result without the protocol that produced it have not been given a fully specified result ^[11].

What a patching result licenses, stated precisely: a positive score licenses the claim that substituting this component’s activation between these two specific runs changes this specific behavioural metric by this specific amount. It licenses a causal claim, which is genuinely stronger than anything probing or dictionary reconstruction can offer — but it is a causal claim indexed to a protocol, not a free-standing description of the network’s mechanism.

Representation engineering and steering vectors: can you move the behaviour by hand

The fourth method asks a more practical question than any of the first three: not whether a property is present, not whether it decomposes cleanly, not even whether a component is necessary, but whether the model’s behaviour can be deliberately and predictably moved by acting on its internal activations from outside.

Zou and colleagues gave this programme a name and an explicit theoretical stance: representation engineering, which they describe as a top-down approach borrowed in spirit from population-level methods in cognitive neuroscience, treating an aggregate direction across many units — rather than an individual neuron or a hand-traced circuit — as the natural unit of analysis. Their framework separates two operations built on the same extracted direction: reading, using the direction to monitor a model’s internal state, and control, adding a multiple of the direction back into the forward pass to change behaviour, and the paper reports applying both to concepts including honesty, harmlessness, and power-seeking tendencies ^[12]. The specific technique for finding and using a direction to change behaviour, independent of any framing about neuroscience, is more simply stated. Turner and colleagues introduced Activation Addition, computing a steering vector directly from the difference between a model’s activations on a pair of contrasting prompts, with no training or optimisation step at all, and adding it into a chosen layer during generation to bias output toward a target concept; they reported the method transferring from GPT-2 to Llama-13B and GPT-J-6B while largely preserving performance on unrelated inputs ^[13]. Rimsky and colleagues refined this into Contrastive Activation Addition, averaging the activation difference over many matched pairs of positive and negative behavioural examples rather than a single pair, which produces a more stable direction, and evaluated the resulting vectors on Llama 2 Chat across both multiple-choice and open-ended behavioural tasks ^[14].

The construction is a mean-difference direction, added back into the residual stream with a tunable strength at inference time:

where $P$ and $N$ index matched positive and negative examples of the target behaviour, $h_\ell$ is the residual-stream activation at layer $\ell$, and $\alpha$ is a coefficient the operator sets by hand. Nothing in this construction requires understanding why the network represents the concept along this direction, only that adding it produces the intended behavioural shift.

That last property is exactly the source of the method’s documented limitation: a working handle is not an explanation, and it is not even guaranteed to keep working outside the conditions it was measured under. Tan and colleagues ran a systematic reliability study of steering vectors built this way and reported two separate failure patterns. In distribution, the degree to which a given vector actually shifted behaviour was highly variable from one input to the next, with the paper attributing part of that variance to spurious correlations the contrastive construction picks up incidentally rather than the intended concept itself. Out of distribution, steering vectors that worked reliably on the prompts they were built from were, for several of the concepts tested, brittle to prompt changes a person would consider unremarkable, failing to generalise in ways that were not predictable in advance ^[15]. A steering result therefore licenses the claim that adding this vector, at this layer and this coefficient, moved this behaviour on these inputs. It does not license a claim about the concept’s underlying organisation, and its own literature is explicit that the same vector’s effect should not be assumed to travel to a different prompt distribution untested.

What the four measurements do not tell each other

Laid side by side, the four methods form a table of non-overlapping licenses, not a ranked list:

The practical consequence is that combining methods is not redundant, because each closes a gap the others leave open. A probing result that a property is decodable becomes far more informative once a patching experiment shows a component carrying that property is also causally necessary for a downstream behaviour; a sparse-autoencoder feature becomes more trustworthy once a steering intervention on it, rather than merely a reading of it, produces the predicted behavioural change. None of that stacking produces a single number that ranks the four methods against each other, because they were never measuring the same quantity. Reporting only one of the four and describing its result as “what interpretability found” about a model, without naming which of the four questions was actually asked, is the single most common way this field’s findings get overstated in secondary coverage.

It also follows that a negative result from one method does not overturn a positive result from another. A property failing to probe well at a given layer does not mean a steering vector built at a different layer cannot move the associated behaviour; a sparse autoencoder failing to isolate a clean feature for a concept does not mean that concept lacks a causally necessary component findable by patching. Each method has its own denominator, and comparing across denominators produces exactly the kind of misleading cross-method ranking that a careful reading of any one of these papers argues against.

Predictions, with what would falsify them

These are forecasts, kept separate from the sourced findings above. Horizon: 12 August 2029. Assumptions: continued publication of interpretability methodology by both academic groups and frontier laboratories, no architectural shift away from the transformer as the dominant substrate, and no regulatory mandate forcing a single standardised interpretability audit.

One. Published interpretability results will increasingly report which of the four questions was asked, rather than a single unqualified claim about what “was found” in a model, following the pattern already visible in the methodological papers this article draws on. Disconfirmed if leading venues in 2029 still routinely publish headline interpretability claims without specifying probing, dictionary-reconstruction, causal, or steering evidence as the basis.

Two. Combined pipelines — a dictionary-derived feature validated by patching or by steering, rather than by reconstruction quality alone — will become the default reporting standard for a new interpretability claim, displacing single-method demonstrations. Disconfirmed if the majority of new feature-level claims in 2029 still rest on reconstruction and automated-interpretability scoring alone.

Three. Documented failure modes specific to each method — probe overcapacity, feature absorption or its successors, protocol-dependent patching scores, and steering brittleness out of distribution — will each accumulate a larger literature of follow-up corrections than the founding method paper itself, because finding a method’s failure mode is cheaper than building the method. Disconfirmed if any of the four methods reaches 2029 with its founding paper still cited more than its critiques combined, within its own subfield.

Four. Steering-vector techniques will see operational deployment for behavioural control before either probing or sparse autoencoders see operational deployment for certifying the absence of a property, because control requires only a working handle while certification requires the coverage none of the four methods currently offers. Disconfirmed if a published third-party audit certifies the absence of a specific property in a frontier model primarily on probing or dictionary-reconstruction evidence.

What to take away

Four methods, four denominators. A probing classifier tells you whether a property is linearly recoverable from one tapped layer, against a control baseline, and nothing about whether the network downstream ever reads it. A sparse autoencoder tells you whether a layer’s activity admits a sparse decomposition in some learned basis, and nothing about whether that basis is the network’s own — feature absorption shows the two can diverge even when reconstruction looks clean. Activation patching and causal tracing tell you whether substituting one component’s activation between two specific runs changes a specific behavioural metric by a specific amount, under a specific protocol that must be reported alongside the result to mean anything. Representation engineering and steering vectors tell you whether adding a direction back into the forward pass moves behaviour predictably at a chosen layer and strength, on the inputs actually tested, with no guarantee the same vector travels to a different prompt distribution untried.

None of the four is an upgrade on the others, and none of the four’s papers claims to be. The question to ask of any interpretability result is not “does this method work” but “which of these four questions did it answer, and does the claim built on top of it match that answer” — because the most common way these results get overstated is not fabrication, it is quietly swapping one denominator for another between the paper and the sentence describing it.