superscript

The passage around this formula

Minimizing over T gives an optimal interval $T^{*}$ = $\sqrt{2\tau/\lambda}$ : the checkpoint interval should shrink as the inverse square root of the fleet failure rate, not linearly with it, so frequency rises roughly as $\sqrt{N}$ purely from the failure-rate side as accelerator count N grows. Comparing that to MLPerf’s own reported tightening — interval falling from 9.3 to 1.5 minutes, a 6.2-times increase in frequency, as scale rises 6.25-times from 16,000 to 100,000 accelerators — the real tightening is close to linear in scale, noticeably steeper than the $\sqrt{N}$ this simple model predicts. That gap is informative rather than a contradiction: it is consistent with operators holding a fixed…

Read this part in the article →