addition

The passage around this formula

The cost is measurable and it is not small, even for a genuinely well-built classifier, and the reason is arithmetic rather than a flaw in any particular model. Suppose a classifier catches 98% of genuinely harmful requests and wrongly flags only 1% of benign ones — numbers that would read, on a slide, as a strong result. If the true base rate of harmful requests in real traffic is low, say one in two thousand, which is the ordinary case for a general-purpose product, precision — the share of flagged requests that are actually harmful — is $\text{Precision} = \frac{\mathrm{TPR} \cdot \pi}{\mathrm{TPR} \cdot \pi + \mathrm{FPR} \cdot (1 - \pi)}$. where $\mathrm{TPR}$ and $\mathrm{FPR}$ are the true- and false-positive rates and $\pi$ is the base rate. Plugging in $\mathrm{TPR}$ = 0.98…

Read this part in the article →