← Back to the equation

Equation field guide

Scale factor S

The scale S is the size of one step on the decoded number line. It must be positive.

A ruler for the integer codes

If S = 0.25, codes one apart represent values 0.25 apart. For Z = 10, code q = 10 means 0, q = 11 means 0.25, and q = 14 means 1.0.

Multiplying q − Z by S changes an integer step count into the model’s numerical scale. S is chosen during quantization or calibration; it is not recalculated from each individual code.

Precision and range

A smaller S puts representable points closer together, which reduces rounding error near those points. With a fixed number of integer codes it also narrows the total real-value interval. A larger S spreads the points farther apart and covers more range.

Sources and further reading