Speech Enhancement and Noise Suppression

A compliance recorder applied aggressive enhancement to calls before archiving them. The output sounded clear, which was the point of installing it. What the archive no longer held was the recording. Quiet names had been altered. Some non-speech cues had disappeared. No later forensic review could separate what the microphone had captured from what the model had reconstructed.

The uncomfortable part is that a release review would have signed this off. Ask only about intelligibility and transcript change on realistic slices and the recorder passes. What was lost is not in the score. It is in the archive. Every claim this lesson makes about that failure has a published counterpart, with a name, a date and a number attached. The rest of the lesson is those counterparts.

A single-channel denoiser has one mixture and nothing else. Whatever it cannot take from the recording it has to take from learned or statistical structure about how sources behave. Multi-channel enhancement has more than one observation of the same scene. A downstream front end is not obliged to produce listenable audio at all, only representations good enough for the task sitting behind it.

These are three different questions wearing one name. What clears one of them clears neither of the others. A result quoted from one setting says nothing about the other two. The named experiments in the rest of this lesson are almost all single-channel experiments. That is the hardest of the three cases. It is also the one the compliance recorder was in.

Speech enhancement estimates a cleaner target from a noisy recording. Wherever noise and speech overlap in the same time–frequency region, the mixture does not determine the answer. The task is underdetermined there. Something other than the observation has to fill the gap. That is why none of the three can simply recover the original.

Systems fill it differently. One predicts a mask, another a spectrum, another a waveform, another a residual. Those look like architectural choices and mostly are not. A team should weigh single-channel denoising, multi-channel enhancement and a downstream front end against each other first. The architecture follows from that decision rather than driving it.

One property is worth holding on to for the rest of the lesson. Enhanced audio is an estimate, not a recovered original. An estimate can be wrong while sounding entirely plausible: a word deleted, musical noise, phase distortion, a voice that is no longer quite the speaker's, or synthetic speech-like components sitting where the speech used to be.

Three things have to happen in order. Specify the mixture conditions the system will actually meet. Define the target signal it is supposed to recover. Preserve auditability, so the untouched recording can still be inspected afterwards. The recorder never met the third.

That third step is not a principle this lesson invented. It is written down twice. ENFSI's forensic speech and audio group states it as a requirement in its Best Practice Manual for Digital Audio Authenticity Analysis, approved in December 2022. Section 8.2, “In the Laboratory”, reads: “Practitioners should always create and work on a working copy of the audio evidence, and not on the original submitted copy.” The same section requires the working copy to be hash-verified with SHA-2 or SHA-3. Access to the evidence storage has to be protected by write blockers.

The second body says it independently, and says it about enhancement specifically. SWGDE's Best Practices for Enhancement of Digital Audio, released in August 2025, requires every examination to be carried out on a working copy. Section 10.1 requires the processes, settings and applications used to be documented “in sufficient detail to repeat or reproduce the final results”, with their versions and the time segments processed. That revision's history records Appendix A being revised for the inclusion of “deep learning filters (trained neural networks)”. The forensic community has already written the neural denoiser into its documentation requirement.

Under time pressure it is the first step that goes. Nobody writes the mixture conditions down, so the definition of the target is never tested against them. It does not stop being used for that. It travels down the entire chain to the moment auditability matters, and by then it has been carried so long that nobody remembers it was ever a choice. The compliance recorder did not fall short of an ideal. It archived the enhanced audio in place of the original, which two published standards forbid.

Part of the recorder's trouble is that enhancement is not one product. It is deployed in at least four places that disagree about how much suppression is acceptable. Each of them pairs its own unit with its own thing to demonstrate.

The ASR case has been taken apart in detail, and the mechanism is not the obvious one. Researchers at NTT and Doshisha University split the error of a single-channel enhancement system into two parts. One is the noise it failed to remove. The other is an “artifact” component: the part of the error that cannot be written as a linear combination of the speech and noise sources. They separated the two by orthogonal projection-based decomposition. Their 2022 finding is blunt: “We experimentally identify the artifact component as the main cause of performance degradation, and we find that mitigating the artifact can greatly improve ASR performance.” The remedy is almost comic in its directness. Iwamoto and colleagues added a scaled copy of the observed signal back into the enhanced signal. The signal-to-artifact ratio rose, and ASR improved on both simulated and real recordings. What breaks the recognizer is not the noise the denoiser left behind. It is what the denoiser added.

Forensic provenance depends on describing what happened to the audio accurately. Two of the words in common use sound like siblings and do very different jobs. Musical noise is an artifact with an agreeable name. Speech distortion is damage to the very thing the system was installed to rescue.

A time–frequency mask and a perceptual loss are the tools that produce both, which is how the confusion starts. Call speech distortion "musical noise" in a report and you have described damage as an artifact. Different proof, different unit, and often a different person deciding what to do about it. The artifact decomposition is the precise version of this distinction. It separates what the enhancement failed to remove from what the enhancement itself put there. Only the second one is the artifact component.

Naming the damage correctly is what lets an evaluation catch it. The standards bodies and the large public challenges have both already concluded that one number will not do it.

The ITU has a recommendation written specifically for judging noise suppressors, and its design is the argument. ITU-T P.835 has been in force since 2003 and was last revised in July 2026. It does not ask listeners for a quality score. It asks them for three: the speech signal (SIG), the background noise (BAK) and overall quality (OVRL), rated separately. Microsoft researchers put it plainly — the “ITU-T Rec. P.835 subjective evaluation framework gives the standalone quality scores of speech and background noise in addition to the overall quality.” The official listening test was built on the assumption that a suppressor can clean the background while damaging the speech, and that one number would hide it.

The URGENT 2024 Speech Enhancement Challenge reached the same place from the other direction, at scale. Its 21 finalist teams were scored on 13 evaluation metrics across five categories: non-intrusive SE metrics, intrusive SE metrics, downstream-task-independent metrics, downstream-task-dependent metrics, and the subjective metric MOS. The MOS component ran as ITU-T P.808 tests on a 300-sample subset of the blind test data, half simulated and half real-recorded. Eight vetted listeners rated each sample. The organisers' own conclusion, published in 2025: “On the other hand, using solely a single category of metrics, especially non-intrusive metrics, for SE performance assessment can be unreliable or even misleading.” Combining the families tracked human judgement. Leaning on one did not.

So when intelligibility and ASR change disagree with artifact and original-versus-enhanced review, that conflict is not noise in the process. It is what the evaluation is run to produce. It should mark a decision point. The moment a word goes missing while the audio improves, the system is obliged to keep the original and fall back to it.

An evidence matrix is how that comparison stops depending on whoever happens to be listening. It has one job: to let another team challenge a score that improved while a word went missing. That is the challenge nobody brought against the compliance recorder.

What specifying mixture conditions actually costs is on the record. The INTERSPEECH 2020 Deep Noise Suppression Challenge, run by Microsoft, built its training speech out of LibriVox chapters rated with ITU-T P.808. It kept the upper quartile: chapters with 4.3 ≤ MOS ≤ 5. That came to 500 hours of speech from 2150 speakers. The noise side was about 150 audio classes and 60,000 clips drawn from AudioSet and Freesound, augmented with a further 10,000 clips from Freesound and DEMAND. That is what the second step of this process looks like when someone does it properly rather than naming it.

The test set is the part worth copying. The organisers released four categories of 300 clips each: synthetic without reverb, synthetic with reverb, real recordings collected internally at Microsoft, and real recordings from AudioSet. The reason is in their own abstract — “However, often the model performance degrades significantly on real recordings.” A synthetic test set drawn from the training distribution flatters the model. The split exists so the gap is visible instead of averaged away.

Three things then go on your page. Write down what defining the target signal assumes, since that is the assumption shown earlier travelling untested down the chain. Add one counterexample, a case where the definition would mislead you. Then say what preserving auditability triggers when the two disagree: who is told, what is kept, and which version becomes the record.