Equation 2 · Part 3 · What Actually Happens Inside a Very Long Claude Context Window
Symbol O
What this part means
O is one factor in the product that computes the quantity on the left.
Its job in the formula
O is one factor in the product that computes the quantity on the left.
Full expression→Symbol O→Article meaning
The passage around this formula
Anthropic’s own engineering guidance, published in September 2025 as advice for developers building long-running agents, gives the clearest available first-party account of why context rot happens architecturally rather than treating it as an unexplained empirical curiosity [ 4 ] . The explanation rests on the transformer’s core mechanism: every token attends to every other token in the context through self-attention, so the number of pairwise relationships the model must represent grows with the square of the sequence length. For a context of n tokens, the compute spent by self-attention within a single layer scales as . where d is the model’s hidden dimension. Doubling…
Learn the underlying idea
A function assigns an output to each allowed input. The expression f(x) means “apply f to x”.
Open the illustrated functions: inputs become outputs guide →
See this notation across published equations →
Sources cited in the surrounding passage
These citations provide research context; check each source for the exact claim it supports.