Time–Frequency Analysis: Fourier Transform, STFT, and Windowing

Machinery diagnosis has a public reference recording, and its settings are published rather than described. The Case Western Reserve University Bearing Data Center seeded its faults by electro-discharge machining, from 0.007 to 0.040 inches in diameter. It ran the motor at loads of 0 to 3 hp, at speeds from 1797 down to 1720 rpm. And it fixed the sampling rate before anything else: “Data was collected at 12,000 samples/second and at 48,000 samples/second for drive end bearing experiments.”

Those numbers are the whole analysis contract. A fault seven thousandths of an inch across announces itself as something brief happening against something steady. Twelve thousand samples a second decides how brief a thing can still be represented at all. The length of the stretch you transform decides whether it survives being averaged with everything around it. Average a long record into a single spectrum and a short repeating impact is spread thin across the whole of it. Put the same record into a time–frequency view and it is obvious. Nothing in the analysis breaks when the impact vanishes. That is the uncomfortable part. The resolution figures come out perfectly respectable and give no sign that anything was lost.

The dataset also shows what it costs to find out. The field had been testing algorithms on it without a benchmark. Smith and Randall built one in 2015: three established diagnostic techniques, applied to the entire data set, every record categorised. The study has been cited 2,643 times. A dataset everyone had already published on still had to be re-examined record by record before its numbers meant anything.

A long window does one thing well. Length buys frequency discrimination: the longer the stretch of signal you transform, the finer the distinctions you can draw between neighbouring tones. It pays for that in time. The longer the window, the more thoroughly a rapid change is spread across it and averaged with everything else inside. A short window makes the opposite bargain. Transients land where they happened, spectral peaks widen, fine frequency detail goes.

Multi-resolution analysis declines to make a single bargain. In broadcast loudness metering, declining is not a design preference. It is a written requirement. EBU Tech 3341, in its 2023 edition, meters the same signal at two window lengths at once: “The Momentary Loudness uses a sliding rectangular time window of length 0.4 s.” Short-term loudness runs alongside it on a sliding rectangular window of 3 s. Both are explicitly ungated. A second body defines the same two scales independently. ITU-R BS.1771-1 uses a first-order low-pass filter with a 400 ms time-constant for Momentary, and an ungated interval of 3 seconds for Short-term.

The ratio matters more than either number. One signal, two events being watched: a sudden jump and a sustained level. No single window answers both, so two run side by side and neither is called the default. None of the three arrangements is safe by inheritance. Each has to be defended on its own numbers, against the event someone is actually trying to see.

Behind that trade sits a mechanism worth stating plainly. The Fourier transform represents a signal using sinusoidal components. The short-time Fourier transform applies that same analysis to overlapping windows, one after another, so the picture is allowed to change as the signal does. Three settings govern the result: the length of the window, its shape, and the hop between consecutive windows. Between them they decide which distinctions in time and in frequency remain visible at all, and how faithfully the representation can be inverted back into a signal.

In practice those settings are not free parameters. For speech, two independent teams wrote down the same ones. Conformer, in 2020: “We extracted 80-channel filterbanks features computed from a 25ms window with a stride of 10ms.” Whisper, three years later: “All audio is re-sampled to 16,000 Hz, and an 80-channel log-magnitude Mel spectrogram representation is computed on 25-millisecond windows with a stride of 10 milliseconds.” Same window, same stride, same 80 channels, different organisations.

Neither figure is a convention. Ten milliseconds gives a hundred frames a second, close to the rate at which phones change. Twenty-five milliseconds is long enough to resolve a pitch period and short enough to treat as stationary. At Whisper's stated 16,000 Hz those durations are concrete: a 400-sample window advanced 160 samples at a time. And the setting delivers. The 118.8M-parameter Conformer (L) reaches 2.1%/4.3% WER on LibriSpeech test-clean/test-other without a language model, and 1.9%/3.9% with one. That is what a window matched to an event looks like when somebody has done the work.

What comes out the other end is still not a photograph of independent physical frequencies. The bins of a spectrogram reflect the analysis basis you chose: a finite window, a sampling rate, a scale, a display transform. Lines and blobs can be artifacts of leakage or of plotting. So two questions decide whether an analysis can be trusted, and neither is about the model. Was the window matched to the event you are trying to see? Were controlled test signals run through the same settings? Without both answers, the resolution numbers cannot support the decision they are quoted for.

The tone, the impulse and the sweep come before the real data. Each has an answer you already know: the frequency of the tone, the moment of the impulse, where the sweep has reached at any instant. So whatever the analysis blurs, shifts or adds shows up as a discrepancy rather than as a discovery.

Step two is where the argument has to be made, and somebody has made it in public. Broadcast loudness measurement does not leave the window and the hop to the implementer. ITU-R BS.1770-5, from November 2023, writes them down: “A gating block is a set of contiguous audio samples of duration Tg = 400 ms, to the nearest sample. The overlap of each gating block shall be 75% of the gating block duration.” Seventy-five per cent overlap of a 400 ms block is a 100 ms hop. The same Recommendation fixes an absolute gate at −70 LKFS, and a relative gate 10 dB below the level measured after the absolute gate. EBU Tech 3341 independently restates the same 400 ms blocks at 75% overlap and the −70 LUFS gate. A window and a hop can be a written obligation, not whatever the library defaulted to.

That discipline only holds if the four moves are kept apart: what was measured, what was modelled, what was decided, and what was verified. Windowing decisions hide easily inside a single score. A resolution figure that was a measurement of the analysis gets read as a verification of the result.

Twenty-five milliseconds at a ten-millisecond stride is an answer. Be exact about the question it answers. Speech, re-sampled to 16,000 Hz, by teams who reasoned from pitch periods and phone rates and then reported the word error rates the setting bought them. Carry it elsewhere and the reasoning does not travel with it. Broadcast loudness runs 400 ms and 3 s at once, because two bodies wrote both down. Machinery is the Case Western Reserve recording from the opening of this lesson, sampled at 12,000 or 48,000 samples per second, where something brief has to be seen against something steady. Spatial audio asks for a quantity the other three can afford to throw away. Each domain sets its own units. None of them is answered by what answered another.

The habit under test is the one that opened the lesson: a window picked by convention instead of by the length of the event. An experiment is worth building only if another team can run it without you and bring that charge for themselves. Smith and Randall did exactly that to a whole community's favourite dataset in 2015 — three established diagnostic techniques, the entire set, every record categorised, rather than an argument.

Three things go on the page. Write down what matching the window to the event duration assumes. Add one counterexample, a case where that matching would mislead you. The broadcast meters are ready-made, since no single window satisfies both the 0.4 s and the 3 s obligation. Then say what a run on controlled test signals tells you afterwards, and what it cannot tell you, because the test runs inside the window you already chose. Send a steady tone, a short impulse and a frequency sweep through several window lengths, overlaps and window functions. Record the smearing, the leakage, the missed transients and the compute. Then choose from how the task behaves, not from how the plot looks.

Go back to the figures that looked fine. Time and frequency resolution under the chosen window is a property of the analysis that was run. It states what that window could in principle have separated. It says nothing at all about whether the window fitted the event. That is why a window picked out of habit still produces respectable numbers.

Leakage, at least, is not a matter of impression. It has a published number, and has had one since 1978. Harris catalogued, window by window, the 3.0-dB main-lobe bandwidth and the highest sidelobe level. The rectangle window's first sidelobe sits roughly 13 dB below the main-lobe peak. The Hamming window with coefficient 0.53856 attains a −43 dB sidelobe level. Thirty decibels of difference, printed in a table, in a paper OpenAlex records at 7,277 citations. So the core figure needs provenance, and it needs company. Name the unit, say which recordings it came from, how uncertain it is, and what was running while it was measured. Then put three more kinds of number beside it.