Equation 2 · Part 1 · How AI Inference Serving Actually Works
Symbol t
What this part means
a position index.
Its job in the formula
t is a part of this expression. Its role is fixed by the surrounding article and by the operations shown in the formula.
Full expression→Symbol t→Article meaning
Where the article explains it
Decode then produces the answer one token at a time, and each step depends on the one before it: the model cannot compute position t+1 until it has sampled position t , so there is no way to parallelize across output positions the way prefill parallelizes across input positions.
The passage around this formula
…at once, as one large, dense matrix multiplication. Decode then produces the answer one token at a time, and each step depends on the one before it: the model cannot compute position t+1 until it has sampled position t , so there is no way to parallelize across output positions the way prefill parallelizes across input positions. The later positions simply do not exist yet.
Learn the underlying idea
A variable is a named place for a value. Its letter is a local label: x can mean position in one formula and a data point in another.
Open the illustrated variables: a letter stands for a value guide →
See this notation across published equations →
Sources cited in the article section
These citations provide research context; check each source for the exact claim it supports.