Symbol Δ

The passage around this formula

The standard instrument is activation patching : run the model on a clean input, run it on a corrupted one, then substitute the activations of a chosen component from one run into the other and measure the change in some behavioural metric. Formally, for a set of components $\mathcal{C}$ and a metric m , $\Delta(\mathcal{C}) = m\!\left(M_{\mathrm{clean}} \text{ with } \mathcal{C} \text{ set to } a_{\mathcal{C}}^{\mathrm{corrupt}}\right) - m\!\left(M_{\mathrm{clean}}\right)$. Meng and colleagues used a version of this — causal tracing — to localise factual recall, finding that a distinct set of steps in middle-layer feed-forward modules mediate factual predictions at the subject token, and then used the localisation to perform rank-one weight edits that changed specific facts [ 14 ] . The edit is the strongest form of the argument: a…

Read this part in the article →