Published equation contexts

$\Theta(n^{0.65})$

Why this formula appears here

Because published TopK configurations keep k in the tens to low hundreds while n runs into the millions, n $\gg$ k and the encoding term dominates almost entirely: $C_{\mathrm{SAE}}$ $\approx$ 2dnT . That single approximation explains something the paper reports without deriving: convergence — the point at which more tokens stop buying lower reconstruction error — is reached later as n grows, empirically as $\Theta$ ( $n^{0.65}$ ) tokens for GPT-4-scale autoencoders [ 1 ] . Cost scales with the product of dictionary width and token count, and pushing width up forces token count up too if the dictionary is to be trained to convergence rather than merely trained. The paper is explicit that this collided…

Read the full article-specific guide →

Read the representative guide

\Theta

Symbol Theta

Theta is a part of this expression. Its role is fixed by the surrounding article and by the operations shown in the formula.

Read this term in its guide →

n^{0.65}

Symbol n^0.65

$n^0$ .65 is a part of this expression. Its role is fixed by the surrounding article and by the operations shown in the formula.

Read this term in its guide →

How to interpret it

Read this expression with the definitions, units, and assumptions supplied by the article.

Research cited beside this formula

Published contexts (1)

A symbol can carry a different meaning in another article. Each occurrence keeps its own guide and term definitions.

\Theta(n^{0.65})

Equation 20 · AI Research

What Interpretability Actually Costs to Do at Scale

This mathematical expression combines the displayed quantities; its precise role follows from the surrounding article text.

Because published TopK configurations keep k in the tens to low hundreds while n runs into the millions, n $\gg$ k and the encoding term dominates almost entirely: $C_{\mathrm{SAE}}$ $\approx$ 2dnT . That single approximation explains something the paper reports without deriving: convergence — the point at which more tokens stop buying lower reconstruction error — is reached later as n grows, empirically as $\Theta$ ( $n^{0.65}$ ) tokens for GPT-4-scale autoencoders [ 1 ] . Cost scales with the product of dictionary width and token count, and pushing width up forces token count up too if the dictionary is to be trained to convergence rather than merely trained. The paper is explicit that this collided…

Equation guide → · Article →

Θ(n0.65)\Theta(n^{0.65})Θ(n0.65)