Published equation contexts
Why this formula appears here
which is the reason ring all-reduce is called bandwidth-optimal: in the limit, total traffic per worker approaches twice the buffer size regardless of how many workers participate, rather than growing with p . What does grow with p is the number of sequential steps, 2(p-1) , and each step’s latency is bounded below by the slowest link and the slowest worker in the ring — which is precisely why NCCL switches to a tree algorithm for smaller messages and larger worker counts, trading some bandwidth efficiency for a p step count instead of a linear one [ 5 ] . This is an analytical property of the algorithm, not a vendor claim; the tree-versus-ring choice NCCL actually makes at runtime,…
Read the representative guide
How to interpret it
Read this expression with the definitions, units, and assumptions supplied by the article.
Research cited beside this formula
Published contexts (2)
A symbol can carry a different meaning in another article. Each occurrence keeps its own guide and term definitions.
Equation 10 · Datacenters
How AI Datacenter Interconnects Actually Work
This mathematical expression combines the displayed quantities; its precise role follows from the surrounding article text.
which is the reason ring all-reduce is called bandwidth-optimal: in the limit, total traffic per worker approaches twice the buffer size regardless of how many workers participate, rather than growing with p . What does grow with p is the number of sequential steps, 2(p-1) , and each step’s latency is bounded below by the slowest link and the slowest worker in the ring — which is precisely why NCCL switches to a tree algorithm for smaller messages and larger worker counts, trading some bandwidth efficiency for a p step count instead of a linear one [ 5 ] . This is an analytical property of the algorithm, not a vendor claim; the tree-versus-ring choice NCCL actually makes at runtime,…
Meanings in this article
Equation guide → · Article →Equation 25 · AI Hardware & Semiconductors
The Network Is the Computer Again
This mathematical expression combines the displayed quantities; its precise role follows from the surrounding article text.
paying a logarithmic latency term but sending the full message in every round, so its bandwidth cost grows with p rather than staying flat.
Equation guide → · Article →