← All parts of this equation

Equation 15 · Part 4 · The Network Is the Computer Again

Symbol α

Tring(n,p)=2(p−1) α+2(p−1)p n β+p−1p n γ.T_{\mathrm{ring}}(n,p) = 2(p-1)\,\alpha + \frac{2(p-1)}{p}\,n\,\beta + \frac{p-1}{p}\,n\,\gamma .
α\alpha

What this part means

the writing.

Its job in the formula

α is one of the signed contributions combined to compute the quantity on the left.

Where the article explains it

Writing α\alpha for latency per message, β\beta for transfer time per byte and γ\gamma for reduction time per byte, the cost of a collective over p participants on n bytes takes the shape T(n,p) = α\alpha\,L(p) + β\beta\,W(n,p) + γ\gamma\,C(n,p), where L counts message steps, W counts bytes crossing each link and C counts arithmetic.

The passage around this formula

Consider the two canonical all-reduce algorithms. A ring arranges participants in a cycle and performs a reduce-scatter followed by an all-gather, each in p-1 steps carrying n/p bytes: Tring(n,p)=2(p−1) α+2(p−1)p n β+p−1p n γT_{\mathrm{ring}}(n,p) = 2(p-1)\,\alpha + \frac{2(p-1)}{p}\,n\,\beta + \frac{p-1}{p}\,n\,\gamma . The bandwidth term converges to 2nβ\beta as p grows — it stops depending on the number of participants — which is why the ring is the bandwidth-optimal choice for large messages. NVIDIA’s own performance documentation encodes exactly this factor, defining the bus bandwidth of an all-reduce by applying a correction of 2(p-1)/p to the naive size-over-time figure, on the reasoning that an all-reduce requires 2(p-1) data transfers across p links; all-gather, reduce-scatter and all-to-all get a…

Read this part in the article →

Learn the underlying idea

A variable is a named place for a value. Its letter is a local label: x can mean position in one formula and a data point in another.

Open the illustrated variables: a letter stands for a value guide →

See this notation across published equations →

Sources cited in the surrounding passage

These citations provide research context; check each source for the exact claim it supports.