Equation 20 · Part 1 · What Interpretability Actually Costs to Do at Scale
Symbol Theta
What this part means
Theta is a part of this expression. Its role is fixed by the surrounding article and by the operations shown in the formula.
Its job in the formula
Theta is a part of this expression. Its role is fixed by the surrounding article and by the operations shown in the formula.
Full expression→Symbol Theta→Article meaning
The passage around this formula
…2dnT . That single approximation explains something the paper reports without deriving: convergence — the point at which more tokens stop buying lower reconstruction error — is reached later as n grows, empirically as () tokens for GPT-4-scale autoencoders [ 1 ] . Cost scales with the product of dictionary width and token count, and pushing width up forces token count up too if the dictionary is to be trained to convergence rather than merely trained. The paper is explicit that this collided with a…
Learn the underlying idea
A function assigns an output to each allowed input. The expression f(x) means “apply f to x”.
Open the illustrated functions: inputs become outputs guide →
See this notation across published equations →
Sources cited in the surrounding passage
These citations provide research context; check each source for the exact claim it supports.