Symbol p
p is a part of this expression. Its role is fixed by the surrounding article and by the operations shown in the formula.
Read this term in its guide →Published equation contexts
Why this formula appears here
Pipeline parallelism splits the model by depth, giving each device a contiguous run of layers. The traffic is point-to-point rather than collective — activations forward, gradients backward, between neighbouring stages only — and the volume is modest. Its cost is not bandwidth but idleness. GPipe’s contribution was to split each batch into micro-batches so that stages could work concurrently instead of waiting for a whole batch to traverse the pipeline [ 6 ] . With p stages and m micro-batches, the standard accounting gives a bubble fraction of . which says the only cure for pipeline idleness is more micro-batches in flight, and micro-batches cost memory. Narayanan and…
Read the representative guide
Symbol m
m occurs below the fraction bar. The quantity above the bar is divided by this expression; zero is excluded as a denominator.
Read this term in its guide →Denominator: m+p-1
The complete quantity below the fraction bar; it must be nonzero for this division.
Read this term in its guide →How to interpret it
With a fixed numerator, increasing a nonzero denominator reduces the fraction.
Research cited beside this formula
Published contexts (1)
A symbol can carry a different meaning in another article. Each occurrence keeps its own guide and term definitions.
Equation 3 · AI Hardware & Semiconductors
The Network Is the Computer Again
This mathematical expression combines the displayed quantities; its precise role follows from the surrounding article text.
Pipeline parallelism splits the model by depth, giving each device a contiguous run of layers. The traffic is point-to-point rather than collective — activations forward, gradients backward, between neighbouring stages only — and the volume is modest. Its cost is not bandwidth but idleness. GPipe’s contribution was to split each batch into micro-batches so that stages could work concurrently instead of waiting for a whole batch to traverse the pipeline [ 6 ] . With p stages and m micro-batches, the standard accounting gives a bubble fraction of . which says the only cure for pipeline idleness is more micro-batches in flight, and micro-batches cost memory. Narayanan and…
Equation guide → · Article →