addition

The passage around this formula

Nanda and colleagues trained small transformers on nothing but modular addition — predicting (a+b) $\bmod$ p for a fixed prime p — a task deliberately chosen because the textbook answer to “what algorithm does this” is fixed and checkable in advance, and studied networks that grok: memorising the training set first, with poor generalisation, and only much later transitioning sharply to a solution that generalises, often long after training loss has already bottomed out [ 15 ] . Extraction and probing came first, applied to the network’s own weights and intermediate activations, and revealed something a probe alone could not have guessed going in: the trained embeddings organised inputs by…

Read this part in the article →