Symbol n
the participant count — specifically so the resulting number can be compared directly against the hardware’s rated peak bandwidth regardless of how many ranks are in the job [ 2 ].
Read this term in its guide →Published equation contexts
Why this formula appears here
That last point is the operational discipline this section is really about: a changed environment variable is a hypothesis, not a result, until it is measured against the fabric it was set on. The standard instrument for that measurement is bus bandwidth as reported by NCCL’s own test suite, which applies an operation-specific correction factor to the raw bytes-per-second figure — 2(n-1)/n for all-reduce, (n-1)/n for reduce-scatter, all-gather and all-to-all, and a factor of one for broadcast and reduce, where n is the participant count — specifically so the resulting number can be compared directly against the hardware’s rated peak bandwidth regardless of how many ranks are in the job [ 2 ]…
Read the representative guide
How to interpret it
Read this expression with the definitions, units, and assumptions supplied by the article.
Research cited beside this formula
Published contexts (1)
A symbol can carry a different meaning in another article. Each occurrence keeps its own guide and term definitions.
Equation 3 · Datacenters
AI Datacenter Interconnects in Practice: An Advanced Technical Guide
This mathematical expression combines the displayed quantities; its precise role follows from the surrounding article text.
That last point is the operational discipline this section is really about: a changed environment variable is a hypothesis, not a result, until it is measured against the fabric it was set on. The standard instrument for that measurement is bus bandwidth as reported by NCCL’s own test suite, which applies an operation-specific correction factor to the raw bytes-per-second figure — 2(n-1)/n for all-reduce, (n-1)/n for reduce-scatter, all-gather and all-to-all, and a factor of one for broadcast and reduce, where n is the participant count — specifically so the resulting number can be compared directly against the hardware’s rated peak bandwidth regardless of how many ranks are in the job [ 2 ]…
Meanings in this article
- : the participant count — specifically so the resulting number can be compared directly against the hardware’s rated peak bandwidth regardless of how many ranks are in the job [ 2 ].