Tagged activation patching · show all articles
A circuit that looks real on your screen has to survive a matched control, a blind check, and someone else's attempt to break it before it earns a place in an audit report. This is how the practitioners who take that seriously actually work.
Four ways of looking inside a trained network answer four different questions — what is decodable, what is the code's basis, what is causally necessary, and what can be moved by hand. None of the four subsumes the others.
Before any claim about what a model is doing, someone ran a forward pass, trained a probe, decomposed an activation into sparse pieces, and changed something on purpose to see what moved. This is that process, in the order it actually happens.