√

The passage around this formula

For a suite of n independent items with true rate p , the standard error of the estimate is $\mathrm{SE} = \sqrt{\frac{p(1-p)}{n}}$. which for p $\approx$ 0.5 and n = 200 is about 3.5 percentage points — before adding any run-to-run generation variance, and before accounting for the fact that benchmark items are not independent. A reported two-point difference between two systems on such a suite is not evidence of anything.

Read this part in the article →