subscript

The passage around this formula

The standard instrument is activation patching. Run the model once on a clean input and once on a deliberately corrupted variant of it — a single token changed, a name swapped — caching every intermediate activation from both runs. Then run the model a third time on the corrupted input, but with one chosen component’s activation overwritten by the value it took during the clean run, and measure the resulting change in some scalar behavioural metric m , typically the gap between the correct answer’s logit and a specific incorrect competitor’s logit: $\mathrm{Patch}(c) = m\Big(M\big(x_{\mathrm{corrupt}};\ a_c \leftarrow a_c^{\mathrm{clean}}\big)\Big) - m\big(M(x_{\mathrm{corrupt}})\big)$. This particular direction — a clean value spliced into an otherwise corrupted run — is called denoising: it measures how…

Read this part in the article →