Spectral Leakage, Scalloping, and Segment Averaging

A tone at a bin frequency of a length-$N$ transform produces a single nonzero coefficient. A tone at any other frequency, however small the offset, produces a coefficient in every bin. That asymmetry is not a numerical defect, and it does not disappear with more precision or better algorithms. It follows from the transform’s assumption that the record is one period of a periodic signal, and from the shape of the window implicitly applied by taking a finite record.

The transform sees a periodic extension

The DFT is defined on a finite sequence, and by construction it represents that sequence repeated forever. If the record contains exactly $k$ cycles of a tone for integer $k$, the repetition has no discontinuity and the result is one bin. If it contains $k + \epsilon$ cycles, the repeated record has a jump at every boundary of size proportional to the amplitude and to the fractional excess. That jump is a step in the periodic extension, and a step has energy at every frequency.

The spectrum of the periodic extension is therefore the spectrum of the tone plus the spectrum of a periodic train of steps, and the second term is what spreads the energy. The coefficients are projections onto a fixed frequency grid, as What the Discrete Fourier Transform Computes sets out, so a component between grid points has no coefficient of its own. The magnitude at the adjacent bins falls off along the sidelobes of the Dirichlet kernel, with an envelope proportional to $1/(\pi\Delta)$ at large distances from the tone, where $\Delta$ is the offset in bins. Twenty bins away the envelope is down to roughly $1/63$ of the peak, about $-36$ dB, and the individual bins between the nulls reach values on that order, so a strong tone can hide a weak one several bins away. This is the mechanism by which a large interfering signal obscures a small one in a spectrum, and it is the reason a spectrum measured without a window is only readable when the record contains a whole number of cycles of every significant component.

Making the record contain whole cycles is possible when the signal has a known fundamental and the sample rate is adjustable. For a fixed sample rate it requires resampling to a length that is an integer number of cycles, which is a nontrivial operation and changes the noise statistics. The general solution is to accept the discontinuity and taper the record toward zero at both ends.

The window trades a wider main lobe for lower sidelobes

Multiplying by a taper reduces the jump at the periodic boundary, and the reduction in the jump reduces the sidelobe level. The cost is the same one that appears in filter design: a taper narrows the effective aperture, so the main lobe widens.

In spectral terms the main lobe’s half-width in bins is the measure of how far a tone’s energy spreads. For a rectangular window it is one bin; for Hann it is two; for a four-term Blackman-Harris it is four. In exchange the first sidelobe falls from $-13$ dB to $-31$, $-61$, or about $-92$ dB respectively. The sidelobe rolloff rate also changes: about 6 dB per octave for rectangular, 18 dB per octave for Hann, and steeper for the multi-term windows.

The engineering decision is which of these two numbers binds. For detecting a small tone near a large one, the sidelobe level at the relevant bin separation is what matters, and a wide window with deep sidelobes wins. For resolving two tones of comparable amplitude that are close together, the main lobe width is what matters, and a narrow window wins. No single window optimizes both; the uncertainty between time and frequency locality is a property of the transform, not a limitation of a particular taper.

There is a third quantity that is often overlooked: the window’s coherent gain, the sum of its samples divided by the number of samples. A rectangular window has coherent gain exactly 1, so a tone’s peak coefficient equals its amplitude times $N/2$. Hann has coherent gain 0.5, so the same tone produces half the coefficient. Correctly scaled spectra divide by the coherent gain, and spectra that are not so divided differ between windows by a factor of two or more. Comparing amplitudes from two analyses that used different windows without normalizing is the most common arithmetic error in spectral work.

Scalloping is the loss between bins

A tone that falls exactly on a bin produces a peak coefficient proportional to the window’s coherent gain. A tone midway between two bins produces two coefficients, each of which is smaller than the on-bin peak. The ratio of the largest coefficient for an arbitrary tone to the largest coefficient for an on-bin tone is the scalloping loss, and its minimum over the offset range is a fixed property of the window.

For a rectangular window the loss at the midpoint is 3.92 dB, close to the $-3.92$ dB value of $|\mathrm{sinc}(0.5)|$. For Hann it is 1.42 dB, and for a flat-top window designed for the purpose it is on the order of 0.01 dB. Flat-top windows achieve that by widening the main lobe substantially and accepting a much lower resolution: the lobe is several bins wide, so two tones closer than that cannot be separated no matter their amplitudes.

Amplitude accuracy and resolution are therefore in direct conflict, and the choice depends on which the measurement needs. A calibration measurement of a known tone’s level wants a flat-top window, because the tone’s frequency will not land on a bin. A search for closely spaced components wants a narrow window and accepts the amplitude error, or interpolates the peak location from the ratio of adjacent coefficients.

import numpy as np

n = 1024

def peak_loss(window):
    w = np.asarray(window, dtype=float)
    w = w / w.sum()
    # On-bin tone: full coherent gain. Half-bin offset:
    k = np.arange(n)
    half = np.exp(2j * np.pi * 0.5 * k / n)
    return 20 * np.log10(abs((w * half).sum()))

print(peak_loss(np.ones(n)), peak_loss(np.hanning(n)))

The two printed values are close to $-3.9$ dB and $-1.4$ dB, matching the documented worst-case scalloping loss for a rectangular and a Hann window respectively. The comparison matters when two analyses of the same tone use different windows: the on-bin amplitude is the same after coherent-gain normalization, but the reported peak differs by the scalloping loss whenever the tone sits between bins.

Interpolation of the peak position is a cheaper alternative to a flat-top window when only frequency needs to be accurate. With a Hann window, the ratio of the two largest adjacent coefficients determines the fractional offset through a closed-form expression derived from the window’s transform, and the amplitude is corrected by the known scalloping at that offset. The method is exact for a single tone in no noise and degrades gracefully as noise is added, since the ratio becomes noisy when the two coefficients are of similar size.

Averaging segments reduces variance, not bias

A single periodogram of a noisy record has a standard deviation comparable to its mean: the estimate of power at each bin fluctuates by roughly the value being estimated, regardless of record length. Increasing $N$ adds bins but does not make each bin’s estimate more stable, because the transform’s orthogonality means each coefficient is computed from an independent combination of the noise, and no averaging across bins occurs.

The remedy is to split the record into $K$ segments, compute a periodogram of each, and average. If the segments are statistically independent the variance of the average falls as $1/K$, so the standard deviation falls as $1/\sqrt{K}$. Overlapping the segments recovers some of the record length lost to splitting; with 50 percent overlap and a Hann window, adjacent segments are only weakly correlated, so the effective number of independent averages is somewhat below the number of segments rather than equal to it. The Welch method formalizes this arrangement, and it is the default in most spectral estimation libraries.

The averaging reduces variance but does not remove bias. Windowing each segment applies the taper twice in effect, since the segment boundaries are tapered and the segment length sets the resolution: the resolution bandwidth is proportional to the window’s main lobe width divided by the segment duration, so shorter segments mean coarser resolution. A record of $L$ samples split into $K$ segments of length $N = L/K$ has a resolution bandwidth $K$ times wider than the transform of the whole record. The variance improvement and the resolution loss are linked through the same parameter, and choosing the segment length is the central decision in the method.

import numpy as np
from scipy import signal

fs = 1000.0
rng = np.random.default_rng(2)
x = np.sin(2 * np.pi * 100.0 * np.arange(20000) / fs) + 0.1 * rng.standard_normal(20000)

for nperseg in (256, 1024, 4096):
    f, p = signal.welch(x, fs=fs, nperseg=nperseg, noverlap=nperseg // 2)
    peak = f[np.argmax(p)]
    print(nperseg, peak, len(f))

The peak location should land near 100 Hz in each case, but the bin spacing narrows as the segment grows, and the number of averages falls correspondingly. That trade is the entire content of the method.

Averaging also assumes the signal’s statistics are stationary across the record. If the amplitude of a component changes during the record, the averaged spectrum is the average of the component’s power over the segments, which is a different quantity from the instantaneous spectrum. Rapidly changing signals require either shorter segments, which cost resolution, or a time-frequency representation that preserves the joint distribution, at the cost of the uncertainty-limited resolution in each coordinate.

There is a further subtlety in the normalization of a power spectral density. Dividing the squared magnitude by the sample rate and by the window’s energy (the sum of squared samples, not the square of the sum) yields a density whose integral over frequency equals the mean square value of the signal. Using the coherent gain instead of the energy gain produces a scale error that depends on both the window and the segment length, and the error is not a constant factor across windows. A spectrum that is correct in shape but wrong in absolute level is usually a normalization mismatch of this kind, and the check is to integrate the result and compare it against the variance computed directly in the time domain.

Finally, the resolution limit is not the same as the ability to detect. A tone whose power is below the noise level in its bin can still be detected if its energy is coherent across segments, because the signal adds in amplitude while the noise adds in power; averaging $K$ segments improves the signal-to-noise ratio of a coherent component by roughly $10\log_{10} K$ relative to a single segment. This is the principle behind coherent averaging of repeated measurements and the reason a periodic component can be recovered from data whose single-record spectrum shows no peak. The same logic underlies the construction of the analytic signal used to track a component’s phase, described in The Analytic Signal and Instantaneous Frequency.