Skip to content
AI.info

Speech and audio

Source Separation: Masks, Waveforms, and Permutation

Cover time–frequency masking, waveform separation, permutation-invariant training, unknown source counts, and faithful evaluation.

By the end you can

Example

The separator produced two clean voices—and swapped them every few seconds

A two-speaker mixture can come back intelligible on both channels and still not come back as two speakers. The output identities permute across windows. A channel carries one voice for a while, then the other. Every speaker-level transcript built on top of that is incoherent.

The cost function is why. Frame-level permutation invariant training assigns each output channel to whichever reference fits best inside that frame. Nothing in that cost carries the assignment from one frame to the next.

This is not hypothetical, and it is not new. The leftover ordering problem was named and removed in 2017, by Kolbæk and colleagues, and their abstract states the mechanism exactly: “uPIT extends the recently proposed Permutation Invariant Training (PIT) technique with an utterance-level cost function, hence eliminating the need for solving an additional permutation problem during inference, which is otherwise required by frame-level PIT.” Make the cost span the utterance, and the frames belonging to one speaker are forced onto one output stream.

The uncomfortable part is what happens before that fix is applied. The metric sides with the audio rather than with the transcript. Both channels score well, because the score is computed window by window. It never asks whether the same voice stayed on the same channel.

  • That is what this lesson is about — time–frequency masking, waveform separation, permutation-invariant training, unknown source counts, and what a faithful evaluation of any of them has to show.
  • One failure produced the whole result: the metric improved while the outputs traded identities. That is the residual inference-time permutation problem uPIT was written in 2017 to eliminate.
  • The evidence anyone would have thought to ask for is scale-invariant signal-to-distortion improvement, quoted with its caveats — the kind of evidence a frame-level system has no trouble producing.
  • The practical response is to stop testing on the mixture the system was built for. Vary the source count, the similarity between voices, their levels, how much they overlap, the room, and classes the model has never met.

Plausible output, not the physical original

Intelligible audio and incoherent transcripts is not a contradiction. The reason sits in what the task actually is. Source separation estimates multiple latent signals from a mixture. Estimates them — that is the whole difficulty. Time–frequency systems predict masks or spectra. Waveform systems work directly in time. Permutation-invariant objectives handle the fact that the outputs arrive in no particular order during training, and, once the cost is defined over the whole utterance rather than the frame, during inference too.

None of those choices gets round the underlying constraint. A mixture can be underdetermined: similar sources, reverberation, clipping, an unknown source count, or content that is correlated rather than independent. Where the recording does not determine the answer, something else fills the gap. What comes out is a plausible separated waveform. It is not necessarily the physical original.

That leaves two questions that sound like one. How much distortion the separation removed is the first. How the result sounds, and what artifacts it carries, is the second. Luo and Mesgarani kept them apart deliberately, evaluating Conv-TasNet by “both objective distortion measures and subjective quality assessment by human listeners”. Neither answer stands in for the other. Neither one alone is a release decision.

A convincing separation can be an invention, not a recovery.

Case

The permutation problem, solved twice

The ordering problem is an old one. It was attacked twice at training time, in two different representations.

Deep clustering stayed in the spectrogram. It assigns a contrastive embedding vector to each time-frequency region, then clusters those embeddings at test time. Hershey and colleagues reported it in 2016, and their own summary of the result is careful: “Preliminary experiments on single channel mixtures from multiple speakers show that a speaker-independent model trained on two-speaker mixtures can improve signal quality for mixtures of held-out speakers by an average of 6dB.” Note what is being claimed and what is not. A speaker-independent model, trained on two-speaker mixtures, measured on held-out speakers.

Conv-TasNet abandoned the spectrogram. It separates in the time domain instead, over a learned linear encoder of the waveform. Luo and Mesgarani published it in 2018, and their claim is the sharper one: “Additionally, Conv-TasNet surpasses several ideal time-frequency magnitude masks in two-speaker speech separation as evaluated by both objective distortion measures and subjective quality assessment by human listeners.”

That second sentence is worth pausing on. An ideal mask is the ceiling of the older formulation. Beating it means the formulation was the limit. And the beating was measured twice, by machine and by ear.

Example

Source count is an assumption first

Both of those results act on ordering while the model is being trained. The other route is to name what you want and pull that one source out. Permutation-invariant training and conditioned extraction are the two ways round the ordering problem.

Conditioned extraction escapes more than ordering. VoiceFilter masks the spectrogram under a speaker condition. Its ten authors, at Google, state the payoff plainly: “Such a system is more applicable to real scenarios because it does not require prior knowledge about the number of speakers and avoids the permutation problem.” They also priced it in the unit that matters downstream. On two-speaker LibriSpeech mixtures, word error rate fell from 55.9% without VoiceFilter to 23.4% with the bi-LSTM model. Clean-speech WER moved only from 10.9% to 11.1%.

Neither route settles how many sources there were to begin with. Conditioning merely declines to ask. Source count is an assumption before it is a number. Source leakage is the residue left when that assumption is wrong.

  • Permutation-invariant training is an objective that chooses the best assignment between unordered model outputs and references — at the frame level, or, after uPIT, across the whole utterance.
  • Source leakage is residual energy from one source showing up in another output.
  • Conditioned extraction separates a target source using an additional identity or content condition. That is why VoiceFilter “does not require prior knowledge about the number of speakers and avoids the permutation problem”, at a cost of clean-speech word error rate rising from 10.9% to 11.1%.
  • Source count is the number of active latent sources the system represents — an assumption baked in before any score is computed on top of it.

Example

Four uses that disagree about what separation is for

How much a wrong source count costs depends on who is asking, because separation is not one product. It is deployed in at least four places that disagree about what a good output even is. Each pairs its own units with its own thing to demonstrate. The transcription failure that opened this lesson belongs to the first of them, where keeping a voice attached to a channel is the entire point. The last two decide the inventory question in opposite directions.

  • In meeting transcription, separation can recover overlapping speech before ASR and diarization ever see it. It is measured there in ASR terms — VoiceFilter's fall from 55.9% to 23.4% WER on two-speaker LibriSpeech mixtures is that measurement.
  • Hearing assistance has to extract a target without suppressing the unexpected sound that mattered.
  • Music production gets remixable stems from a fixed inventory. MUSDB18, released in December 2017, is “a dataset of 150 full lengths music tracks (~10h duration) of different genres along with their isolated drums, bass, vocals and others stems” — 100 training tracks and 50 test, stereo at 44.1 kHz. It was the official dataset of the SiSEC 2018 professionally-produced music recordings task. Four stems is the whole list any score on it can measure against.
  • In bioacoustics the assumption breaks the other way. A 2021 paper trained a MixIT model on birdsong, “outperforming a general audio separation model by over 5 dB in SI-SNR improvement of reconstructed mixtures”. It then showed downstream precision gains for a multi-species classifier across three independent datasets. That is the two-layer evidence overlapping species and environmental noise demand.

Visual

Keep the same voice on the same output

Which is why the order of the decisions matters more than it looks. Three things happen in sequence: define the source inventory, choose a representation and an objective, then evaluate the sources against the use case.

The middle step quietly decides everything after it. Once a representation and an objective are settled on top of an inventory, the inventory stops being a question. The evaluation at the end measures how well the system separated the sources it was told to expect. It never asks whether that list was right.

MUSDB18 is the clean illustration. Drums, bass, vocals and other were chosen in December 2017, before any of the systems scored on the corpus existed. Every number computed against those 150 tracks is a measurement against that list, not a reopening of it.

FigureLayers · 4 layers
  1. 01

    Define the source inventory

    Specify target classes, maximum count, overlap, silence, and unknown-source behavior.

  2. 02

    Choose representation and objective

    Select masks, complex spectra, waveforms, or learned codecs with ordering policy.

  3. 03

    Resolve assignment over time

    Use permutation-invariant training, tracing, or speaker embeddings to maintain consistency.

  4. 04

    Evaluate sources and use case

    Combine signal metrics, perceptual review, identity continuity, and downstream task results.

Your source inventory stops being a decision once representation and objective are chosen on top of it. The final evaluation never reopens the list.

Steps

Audit a separator beyond one score

Reopening the list is what the exercise is for. A separator audited on one number is a separator nobody has audited. Bring the challenge frame-level PIT could not answer: run the exercise so that a second team can take issue with your evidence that each output channel held the same speaker from start to finish.

Three things go on the page. Write down what your list of expected sources assumes — MUSDB18 assumes exactly four stems, and a fixed-count speech model assumes two talkers. Add one mixture that would break it. Then say what you would do differently once the real use case is in front of you, in the unit that use case is consumed in.

FigureProcess · 4 steps

  1. 1. Create mixture families

    Vary source count, similarity, level, overlap, room, and unseen classes.

  2. 2. Track output identity

    Measure channel swaps and segment continuity.

  3. 3. Listen to residuals

    Inspect leakage, missing transients, phase artifacts, and synthetic speech-like errors.

  4. 4. Test downstream use

    Compare recognition, editing, or analysis with and without separation.

Judge the separator by what the recognizer or the editor does with its output, with and without separation in the chain. A score on its own never told you that.

Example

When the metric and the transcript disagree

That challenge is only possible if the evaluation reports more than one thing. A separation number on its own can be made to say almost anything. There is a worked demonstration of that, on a single signal. In 2018 Le Roux and colleagues optimised a mask to minimise SI-SDR on one CHiME2 male utterance. Most of the spectrum went. The result was −4.7 dB SI-SDR. BSS_eval's SDR for that same signal still read 11.6 dB, because BSS_eval is free to fit a filter to the reference that removes similar frequency regions. Their paper is titled SDR – half-baked or well done?, and its verdict on the practice is blunt: “We argue here that the signal-to-distortion ratio (SDR) implemented in the BSS_eval toolkit has generally been improperly used and abused, especially in the case of single-channel separation, resulting in misleading results.”

The same gap opens between any separation number and what consumes its output. Scale-invariant signal-to-distortion improvement will sometimes conflict with downstream ASR, transcription attribution, or music-editing success. One of them says the separation got better. The other says the output stopped being usable. The evaluation exists for exactly the moment they disagree. It should show the conditions under which the separator has to abstain or fall back, including the point where the outputs start trading places.

  • The core evidence for the task itself is scale-invariant signal-to-distortion improvement, quoted with its caveats — and −4.7 dB SI-SDR against 11.6 dB BSS_eval SDR on one utterance is what the caveat is for.
  • Beside it goes the system's behavior — perceptual quality, and a review of the artifacts it leaves behind. That is why Luo and Mesgarani reported human listeners alongside distortion measures.
  • The robustness slice is source-count accuracy, assignment accuracy, and identity continuity. That last one is what frame-level PIT leaves unresolved at inference.
  • Across the life of the system, track downstream ASR, transcription attribution, or music-editing success, since that is what anyone actually consumes. VoiceFilter's 55.9% to 23.4% is that layer, and the birdsong classifier's precision across three independent datasets is the same layer in another field.

Report scale-invariant signal-to-distortion improvement with caveats together with downstream ASR, transcription attribution, or music-editing success.

Key idea

Trained on two voices, deployed on a room

There are four ways this goes wrong, and the top two are enough to sink a broad claim about source separation.

The first is the one this lesson opened with: the score goes up while the outputs trade identities. The second is a model trained on a fixed number of sources and deployed on an open scene. The third is treating silence or a residual channel as though it were a valid source. The fourth is evaluating on synthetic mixtures that omit real overlap, reverberation, and source dependence.

The second and fourth are the stated motivation for an entire training method. Mixture invariant training (MixIT), published in 2020, was built because supervised separation depends on synthetic mixtures resembling real audio. It trains on real single-channel mixtures alone: “In MixIT, training examples are constructed by mixing together existing mixtures, and the model separates them into a variable number of latent sources, such that the separated sources can be remixed to approximate the original mixtures.” A variable number of latent sources is the concrete alternative to a count fixed in advance. It is also the method the birdsong work took up, where the count is never known.

Underdetermination is not an edge case in any of the four. It is what an open room is: similar sources, reverberation, clipping, an unknown number of talkers, and content that is correlated rather than independent. Each of the four is a way that condition gets hidden rather than handled. What the model emits is then one consistent explanation of the mixture. It is not evidence that the room contained it.

A rising score with channels that trade speakers mid-file is worse than a lower score with stable identities, because everything downstream inherits the swap.

Key takeaways