What does this equation mean?

Nanda and colleagues trained small transformers on nothing but modular addition — predicting (a+b) $\bmod$ p for a fixed prime p — a task deliberately chosen because the textbook answer to “what algorithm does this” is fixed and checkable in advance, and studied networks that grok: memorising the training set first, with poor generalisation, and only much later transitioning sharply to a solution that generalises, often long after training loss has already bottomed out [ 15 ] . Extraction and probing came first, applied to the network’s own weights and intermediate activations, and revealed something a probe alone could not have guessed going in: the trained embeddings organised inputs by their residue around a circle, one circle frequency per output logit, rather than in the sparse-lookup structure a memorising solution would use. Decomposing the network’s Fourier-domain activity — a spectral rather than a sparse-dictionary decomposition, but playing the same structural role of breaking a tangled representation into legible parts — showed a circuit computing sums and differences of trigonometric identities across those frequencies, an algorithm with a closed mathematical form that can be checked against the network’s actual weights term by term, not merely inferred from behaviour.