Equation 20 · Cognitive Bias Was the Label; Attention Sink Is the Suspect
What does this equation mean?
Read the formula alongside the article passage below. Each part has a deeper page with its role in the equation, the supporting passage and nearby citations.
This mathematical expression combines the displayed quantities; its precise role follows from the surrounding article text. Read the equation part by part below; each part has a contextual explanation and a link to its mathematical background.
Read it piece by piece
subscript
The lower label selects a particular version, component, or indexed member of the quantity. For example, x₀ and xₜ can be values at different positions.
How to interpret it
Read this expression with the definitions, units, and assumptions supplied by the article.
What the article says around this equation
where is Guo and Vosoughi’s own Jensen–Shannon-divergence effect magnitude, measured by their own published protocol, on an ordinary sink-forming model, and is the identical measurement on an architecture- and data-matched twin trained with sigmoid attention instead of softmax — the exact contrast Gu and colleagues’ released training pipeline was built to produce [ 3 ] . The quantity is dimensionless, bounded in the neighborhood of zero to one for any model pair where the sink-free twin is not somehow more position-sensitive than its sink-forming counterpart, and it answers a single, sharp question: of the effect magnitude the…
Read the full surrounding passage
where is Guo and Vosoughi’s own Jensen–Shannon-divergence effect magnitude, measured by their own published protocol, on an ordinary sink-forming model, and is the identical measurement on an architecture- and data-matched twin trained with sigmoid attention instead of softmax — the exact contrast Gu and colleagues’ released training pipeline was built to produce [ 3 ] . The quantity is dimensionless, bounded in the neighborhood of zero to one for any model pair where the sink-free twin is not somehow more position-sensitive than its sink-forming counterpart, and it answers a single, sharp question: of the effect magnitude the original paper measured, how much of it disappears when the one specific architectural mechanism this article names is removed and nothing else changes?
Sources cited in the surrounding passage
These citations give research context. Read each source to check which claims it supports.
Return to Cognitive Bias Was the Label; Attention Sink Is the Suspect