Quantization Step Size, the Six-Decibel Rule, and Dither

A converter maps a continuous range of input values onto a finite set of output codes. If the range is $R$ volts and the converter has $b$ bits, the step between adjacent codes is $\Delta = R/2^b$, and every input inside a step is reported as the same code. The error is bounded: it can never exceed half a step in magnitude. That bound is the starting point for the usual signal-to-noise figure, but the figure itself comes from treating the error as a random variable, and that treatment is a model with assumptions that can be violated in ordinary use.

The uniform-error model and where the decibels come from

Assume the input moves across many steps during the observation, so that the fractional part of the input divided by the step is effectively uniform on $[0,1)$. Then the error is uniform on $[-\Delta/2, \Delta/2)$, its mean is zero, and its variance is $\Delta^2/12$.

For a full-scale sinusoid of amplitude $R/2$, the mean square value is $R^2/8$. The ratio of signal power to quantization noise power is therefore

$$\frac{R^2/8}{\Delta^2/12} = \frac{3}{2}\cdot\frac{R^2}{\Delta^2} = \frac{3}{2}\cdot 2^{2b}$$

and in decibels, $10\log_{10}(1.5) + 20 b \log_{10} 2 \approx 1.76 + 6.02 b$ dB. That is the origin of the six-decibel rule: one additional bit doubles the number of steps, halves $\Delta$, quarters the noise power, and adds just over 6 dB. Sixteen bits gives about 98 dB, twenty-four bits about 146 dB. The relation is a consequence of the step being a power of two divisor of the range, not an empirical law about converters.

The 1.76 dB constant is specific to a full-scale sinusoid. A signal that occupies the full range but is not sinusoidal has a different crest factor, and a signal that occupies a fraction of the range loses 6 dB for every halving of its amplitude relative to full scale. That last point is the one that bites in practice: an input using a quarter of the converter’s range has thrown away two bits of the achievable ratio regardless of the converter’s resolution. Front-end gain is therefore part of the noise budget, not a convenience.

The same arithmetic applies to any fixed-point representation, which is why it appears whenever floating-point data is stored as integers. Storing a normalized physical quantity in a 16-bit integer field with a known scale factor imposes the same $\Delta^2/12$ variance, and the appropriate question is whether the physical noise already present exceeds it.

When the model fails: the correlated-error regime

The uniform assumption requires the error to behave like an independent random variable. It does not when the input is periodic with a period commensurate with the sample clock, or when it sits near a code boundary and varies by less than a step. In those cases the error is a deterministic function of the input, and its spectrum is not flat: it concentrates at harmonics of the input frequency, including harmonics that fold back into the band of interest.

A slowly varying input near a boundary produces a staircase in the error, and the derivative of that staircase is a train of impulses at the crossing instants. The audible or measurable result is a set of discrete tones rather than a hiss. The signal-to-noise ratio computed from the variance formula still applies, because the total error power is unchanged. What changes is the distribution of that power, and any application that cares about narrowband interference rather than total power is harmed by the concentration.

The model also fails for very low-level signals, when the input spans only a few steps. Then the error is bounded but not uniform, and the quantization is better described as a nonlinear distortion than as additive noise. A signal smaller than one step can vanish entirely, or can appear as a square-wave-like output that toggles between two codes, depending on where it sits relative to the boundaries. The boundary case where the signal sits exactly at half the sample rate is the degenerate one described in Sampling Rate, Aliasing, and the Folding of the Spectrum.

Dither converts distortion into noise

Dither is the addition of a small random or pseudo-random signal before quantization, and its purpose is to break the correlation between input and error. With dither present, the error’s conditional distribution given the input becomes closer to uniform, and the distortion products spread into a noise floor.

The cost is explicit. Adding a dither of peak-to-peak width $\Delta$ (uniform on $[-\Delta/2, \Delta/2)$, zero mean) adds a variance of $\Delta^2/12$, doubling the total noise power and costing 3 dB of signal-to-noise ratio. A dither of peak-to-peak width $\Delta/2$ adds $\Delta^2/48$, a quarter of the quantization variance, at a cost of about 1 dB. The trade is 3 dB of noise for the removal of harmonic distortion; whether that is a good trade depends on whether the application measures total power or spectral purity.

There is a further subtlety with quantizers that round to nearest. Rectangular dither of one step peak-to-peak makes the first moment of the total error independent of the input, which removes the correlated distortion entirely at the cost noted above. Triangular dither, formed by summing two independent rectangular sources and spanning two steps peak to peak, makes both the first and second moments independent, which also removes the modulation of the error variance by the signal. The choice of amplitude and distribution is therefore tied to the order of the moment that must be made input-independent, and specifying “dither” without both is underspecified.

import numpy as np

rng = np.random.default_rng(1)
fs, f0 = 48_000, 997.0
t = np.arange(48_000) / fs
step = 1.0 / 2**16
x = 0.5 * np.sin(2 * np.pi * f0 * t)

def quantize(v, d):
    return np.round((v + d) / step) * step

# Rectangular dither spanning one step peak to peak, zero mean
dither = rng.uniform(-step / 2, step / 2, x.size)
q_dithered = quantize(x, dither)
# Harmonic content of the error collapses; total error power roughly doubles.
err = q_dithered - x
print(np.var(err) / step**2)  # about 1/6, versus 1/12 undithered

The pseudo-random source must be independent of the input and must have enough period that its own pattern does not become a tone. A maximal-length sequence from a linear-feedback shift register is the usual choice, because its period is known and its spectrum is flat to within the sequence’s statistics.

Noise shaping moves the error out of the band

Dither spreads error power evenly. Noise shaping moves it. A feedback quantizer computes the error from the previous sample, filters it, and subtracts the filtered error from the current input before quantizing. The result, in the $z$ domain, is that the output equals the input plus the quantization error passed through $1 - H(z)$, where $H$ is the feedback filter. A highpass $H$ therefore suppresses error in the low band and amplifies it near half the sample rate, and a subsequent digital lowpass removes the amplified part.

x[n] --->(+)--->[ Q ]---> y[n]
          ^                |
          |                v
          +---[ H(z) ]<----+ (error feedback)

First-order shaping with $H(z) = z^{-1}$ gives an error transfer of $1 - z^{-1}$, whose magnitude at low frequency is small and rises toward 2 at $f_s/2$. Raising the order concentrates more of the error near the band edge but risks instability, because the feedback loop includes a nonlinear element; modulators of third order and above require care with input range to avoid overload, and overload changes the transfer characteristics abruptly rather than gracefully.

The classic arrangement combines shaping with oversampling. Because the error power is spread over the full rate and the signal band occupies only $1/M$ of it, decimating by $M$ after shaping retains $1/M$ of the total error power; the filter and phase structure that make that decimation affordable is covered in Polyphase Decimation and the Cost of Changing Sample Rate. With first-order shaping the retained power falls as the cube of the oversampling ratio; with second order, as the fifth power. This is why a one-bit converter clocked at a few megahertz can deliver sixteen-bit resolution in a narrow audio band while a flash converter needs many comparators and a well-matched resistor ladder to reach the same resolution at a much higher bandwidth.

The price of the arrangement is latency and bandwidth. A decimation filter with a sharp transition at a low normalized frequency has a long impulse response, so the group delay is correspondingly large, and the converter’s usable bandwidth is a small fraction of its clock. Successive-approximation and pipeline architectures invert the trade, reaching high resolution at moderate oversampling with short latency and a band that extends close to the Nyquist frequency.

Reading a converter’s noise specification

Datasheets separate two quantities that are often conflated. Signal-to-noise ratio measures all noise and distortion relative to a full-scale input, integrated over a stated bandwidth, usually the Nyquist band. Effective number of bits converts that ratio back through the six-decibel rule to a bit count, and it is always lower than the resolution; a converter that is specified as sixteen bits may deliver fourteen effective bits once its own aperture jitter, thermal noise and nonlinearity are included.

Signal-to-noise-and-distortion differs from signal-to-noise only by including harmonic products, so the gap between the two numbers indicates how much of the degradation is harmonic rather than random. Spurious-free dynamic range measures the largest single tone, which is the number that matters for narrowband detection, and it is usually several decibels better than the total noise figure because a single spur carries less power than the sum of all noise.

None of these figures are improved by processing after the converter. Once an input has crossed a code boundary irreversibly, the information distinguishing the two adjacent values is gone. The only recoverable margins are the ones the analog chain preserves before the first sample, which is why the converter’s noise figure constrains the whole acquisition design rather than just its digital half.