← All parts of this equation

Equation 30 · Part 2 · What an AI Accelerator Actually Is: Silicon, Packaging, and the Memory It Can Reach

Symbol p

Tallreduce≈2 (p−1) α+2 p−1p⋅Nβlink,T_{\mathrm{allreduce}} \approx 2\,(p-1)\,\alpha + 2\,\frac{p-1}{p}\cdot\frac{N}{\beta_{\mathrm{link}}},
pp

What this part means

the latency term grows linearly in.

Its job in the formula

p is a part of this expression. Its role is fixed by the surrounding article and by the operations shown in the formula.

Where the article explains it

The bandwidth term approaches 2N/βlink\beta_{\mathrm{link}} and stops growing with p ; the latency term grows linearly in p .

The passage around this formula

Their cost has a shape worth internalising. For the standard ring formulation of an all-reduce over p devices and N bytes, the operation decomposes into a reduce-scatter followed by an all-gather, each of p - 1 steps in which every device sends N/p bytes. Total time is approximately Tallreduce≈2 (p−1) α+2 p−1p⋅NβlinkT_{\mathrm{allreduce}} \approx 2\,(p-1)\,\alpha + 2\,\frac{p-1}{p}\cdot\frac{N}{\beta_{\mathrm{link}}}. with α\alpha the per-step latency and βlink\beta_{\mathrm{link}} the per-link bandwidth. The…

Read this part in the article →

Learn the underlying idea

A variable is a named place for a value. Its letter is a local label: x can mean position in one formula and a data point in another.

Open the illustrated variables: a letter stands for a value guide →

See this notation across published equations →

Sources cited in the article section

These citations provide research context; check each source for the exact claim it supports.