← Mathematical compendium
Published equation contexts
0.2
Why this formula appears here
I have not measured ϕsink for the checkpoints this paper actually tested, and no claim in this article depends on pretending otherwise. Doing so would require training matched softmax and sigmoid-attention models at Llama2 scale — a run measured in GPU-days this article does not have access to — and then rerunning Guo and Vosoughi’s own classification protocol on both. What can be stated precisely is the protocol and the threshold that would settle the question. Train the matched pair at each of the three Llama2 parameter counts the original paper tested — seven billion, thirteen billion, seventy billion — using the released sail-sg/Attention-Sink pipeline; reimplement Guo and…
Read the full article-specific guide →
How to interpret it
Read this expression with the definitions, units, and assumptions supplied by the article.
Research cited beside this formula
Published contexts (1)
A symbol can carry a different meaning in another article. Each occurrence keeps its own guide and term definitions.
Equation 25 · Language Models & Evaluation
This mathematical expression combines the displayed quantities; its precise role follows from the surrounding article text.
I have not measured ϕsink for the checkpoints this paper actually tested, and no claim in this article depends on pretending otherwise. Doing so would require training matched softmax and sigmoid-attention models at Llama2 scale — a run measured in GPU-days this article does not have access to — and then rerunning Guo and Vosoughi’s own classification protocol on both. What can be stated precisely is the protocol and the threshold that would settle the question. Train the matched pair at each of the three Llama2 parameter counts the original paper tested — seven billion, thirteen billion, seventy billion — using the released sail-sg/Attention-Sink pipeline; reimplement Guo and…
Equation guide → ·
Article →