Equation field guide
Real numbers in a neural network
A real number is any point on the number line. In the quantization equation, r is one numerical model value before it is represented by a compact integer code.
What counts as a real number?
Integers such as −2 and 3, fractions such as 1/4, decimals such as 0.9, and irrational numbers such as √2 are all real numbers. “Real” names the mathematical set; it does not mean “physically real” or “measured without error”.
The line is continuous: there are values between any two distinct points. An 8-bit integer code has only 256 possible patterns, so it cannot represent every real number exactly.
What does r mean in this article?
Here r is a single weight or activation of a neural network. A weight is a learned numerical parameter; an activation is a numerical result produced while the network runs. Both may be negative or fractional.
In actual computer memory, the original model value is normally a finite-precision floating-point approximation. “Real-valued” describes the type of quantity in the model, not a promise of infinite computer precision. Quantization then gives it an even smaller set of representable values.
A concrete example
Suppose the model value is 0.9. With scale S = 0.25 and zero-point Z = 10, the nearest code is q = 14. That code decodes to 0.25 × (14 − 10) = 1.0. The difference, 0.1, is quantization error.