Symbol m

The passage around this formula

…standard errors” on the affected evals [ 8 ] . The general relationship is the one long used in survey statistics: if a naive standard error assumes independence, and trials instead arrive in clusters of average size m with intraclass correlation $\rho$ within a cluster, the correctly inflated standard error is $\mathrm{SE}_{\text{cluster}} = \mathrm{SE}_{\text{naive}} \times \sqrt{1 + (m - 1)\rho}$. Agent trials add a second, harder-to-audit source of the same problem: repeated attempts by one policy on one task instance are not independent draws either, because the same weights,…

Read this part in the article →