√

The passage around this formula

​ purely from the failure-rate side as accelerator count N grows. Comparing that to MLPerf’s own reported tightening — interval falling from 9.3 to 1.5 minutes, a 6.2-times increase in frequency, as scale rises 6.25-times from 16,000 to 100,000 accelerators — the real tightening is close to linear in scale, noticeably steeper than the $\sqrt{N}$ this simple model predicts. That gap is informative rather than a contradiction: it is consistent with operators holding a fixed acceptable progress-loss percentage (MLPerf specifies 5%) rather than freely re-optimizing T against $\tau$ and $\lambda$ jointly, and with $\tau$ itself growing as checkpoints get larger at larger scale — both push the real…

Read this part in the article →