Equation 7 · Part 2 · How AI Datacenter Interconnects Actually Work
subtraction
subtraction
What this part means
Subtract the following term or group from the preceding one. A leading minus marks a negative quantity.
Its job in the formula
Subtract the following term or group from the preceding one. A leading minus marks a negative quantity.
Full expression→subtraction→Article meaning
The passage around this formula
Take the canonical case: a ring all-reduce across p accelerators, each holding a shard of a gradient vector. Mechanically, at the level this article has been building toward, each of the 2(p-1) steps in a ring reduce-scatter-then-all-gather is nothing more exotic than one RDMA WRITE of the kind described above. A collective library such as NCCL posts one work request per hop describing a source range inside the sending GPU’s memory and a destination range inside the next rank’s GPU memory, reachable because both ends registered that memory for GPUDirect RDMA access; the sending NIC’s DMA engine reads the chunk straight out of HBM, frames it as one or more RoCEv2 or native InfiniBand packets,…
Learn the underlying idea
Addition combines quantities; subtraction measures the signed difference between them. Parentheses show what is combined before the rest of the expression is evaluated.
Open the illustrated addition and subtraction in an equation guide →
Sources cited in the article section
- [5] Introduction: NVIDIA Scalable Hierarchical Aggregation and Reduction Protocol (SHARP) ↗
- [6] Scaling Distributed Machine Learning with In-Network Aggregation ↗
These citations provide research context; check each source for the exact claim it supports.