Equation 6 · Part 2 · Probing, Sparse Autoencoders, Patching, and Steering: The Main Interpretability Methods, Compared
subtraction
subtraction
What this part means
Subtract the following term or group from the preceding one. A leading minus marks a negative quantity.
Its job in the formula
Subtract the following term or group from the preceding one. A leading minus marks a negative quantity.
Full expression→subtraction→Article meaning
The passage around this formula
That gap between “predictable from” and “used by” is the method’s central and openly documented weakness. Hewitt and Liang showed that a sufficiently expressive probe can achieve high accuracy predicting properties from representations that plausibly do not encode them in any meaningful sense, because the probe itself has the capacity to memorise idiosyncratic patterns in the training data. Their fix was the control task: construct a version of the labelling scheme that associates each input type with an output at random, so it can only be learned by the probe’s own memorisation capacity, never by any real linguistic signal in the representation. A well-behaved probe should then show high…
Learn the underlying idea
Addition combines quantities; subtraction measures the signed difference between them. Parentheses show what is combined before the rest of the expression is evaluated.
Open the illustrated addition and subtraction in an equation guide →
Sources cited in the surrounding passage
- [3] Designing and Interpreting Probes with Control Tasks ↗
- [4] Probing Classifiers: Promises, Shortcomings, and Advances ↗
These citations provide research context; check each source for the exact claim it supports.