$$\mathbb{E}\!\left[(x - \hat{x})^2\right] \approx \frac{s^2}{12}$$

Equation 31 · Edge AI & Electronics

How a Model Actually Gets Small Enough to Run on a Phone

This equation gives an approximation: it relates the quantities while allowing an approximation.

for a signed integer of b bits. An 8-bit integer has 2^8=256 representable levels; a 4-bit integer has 2^4=16 . That sixteen-fold reduction in available codes is the entire cost of quantization in one number. Treating the rounding error as approximately uniform over one quantization step s , its expected squared magnitude is the classical result $\mathbb{E}\!\left[(x - \hat{x})^2\right] \approx \frac{s^2}{12}$. so halving the number of bits, which roughly doubles s at fixed range, roughly quadruples the expected squared error per weight. That is why INT8 is usually described as close to free and INT4 is not: the error a network has to absorb does not grow gently as bits are removed, it grows quadratically in the step size, and every bit…

Meanings in this article

• $\mathbb{E}$: The expected value operator: the probability-weighted average of the quantity inside its brackets.
• $\hat{x}$: the dequantised approximation actually used in arithmetic.
• $s^2$: the square of s; the scale.

Equation guide → · Article →