← Mathematical compendium

Published equation contexts

ϕsink=1−SPEMsigmoidSPEMsoftmax\phi_{\mathrm{sink}} = 1 - \frac{\mathrm{SPEM}_{\mathrm{sigmoid}}}{\mathrm{SPEM}_{\mathrm{softmax}}}

Why this formula appears here

Everything above is a case for treating the architectural account as at least as well supported as the psychological framing. It is not, on its own, a decisive discriminator, and this article’s one genuine contribution is naming and specifying the measurement that would be. Call it the sink-attributable fraction, ϕsink\phi_{\mathrm{sink}} , defined as ϕsink=1−SPEMsigmoidSPEMsoftmax\phi_{\mathrm{sink}} = 1 - \frac{\mathrm{SPEM}_{\mathrm{sigmoid}}}{\mathrm{SPEM}_{\mathrm{softmax}}}. where SPEMsoftmax\mathrm{SPEM}_{\mathrm{softmax}} is Guo and Vosoughi’s own Jensen–Shannon-divergence effect magnitude, measured by their own published protocol, on an ordinary sink-forming model, and SPEMsigmoid\mathrm{SPEM}_{\mathrm{sigmoid}} is the identical measurement on an architecture- and data-matched twin trained with sigmoid attention…

Read the full article-specific guide →

Read the representative guide

ϕsink\phi_{\mathrm{sink}}

Symbol phi_sink

phisi_sink is part of the quantity the equation computes from the expression on the right.

Read this term in its guide →
SPEMsigmoid\mathrm{SPEM}_{\mathrm{sigmoid}}

Numerator: SPEM_sigmoid

The complete quantity above the fraction bar.

Read this term in its guide →
SPEMsoftmax\mathrm{SPEM}_{\mathrm{softmax}}

Denominator: SPEM_softmax

The complete quantity below the fraction bar; it must be nonzero for this division.

Read this term in its guide →

How to interpret it

With a fixed numerator, increasing a nonzero denominator reduces the fraction. Read it with the definitions, units, and assumptions supplied by the article.

Research cited beside this formula

Learn the mathematics behind it

Published contexts (1)

A symbol can carry a different meaning in another article. Each occurrence keeps its own guide and term definitions.

ϕsink=1−SPEMsigmoidSPEMsoftmax\phi_{\mathrm{sink}} = 1 - \frac{\mathrm{SPEM}_{\mathrm{sigmoid}}}{\mathrm{SPEM}_{\mathrm{softmax}}}

Equation 19 · Language Models & Evaluation

Cognitive Bias Was the Label; Attention Sink Is the Suspect

This equation states an equality: the expressions on both sides have the same value under the article’s assumptions.

Everything above is a case for treating the architectural account as at least as well supported as the psychological framing. It is not, on its own, a decisive discriminator, and this article’s one genuine contribution is naming and specifying the measurement that would be. Call it the sink-attributable fraction, ϕsink\phi_{\mathrm{sink}} , defined as ϕsink=1−SPEMsigmoidSPEMsoftmax\phi_{\mathrm{sink}} = 1 - \frac{\mathrm{SPEM}_{\mathrm{sigmoid}}}{\mathrm{SPEM}_{\mathrm{softmax}}}. where SPEMsoftmax\mathrm{SPEM}_{\mathrm{softmax}} is Guo and Vosoughi’s own Jensen–Shannon-divergence effect magnitude, measured by their own published protocol, on an ordinary sink-forming model, and SPEMsigmoid\mathrm{SPEM}_{\mathrm{sigmoid}} is the identical measurement on an architecture- and data-matched twin trained with sigmoid attention…

Equation guide → · Article →