Voice Biometrics, Privacy, and De-Identification

Three ways of protecting a voice are usually on the table. They are not three versions of one idea. Pseudonymization replaces the direct identifiers and keeps linkability alive on purpose, through a controlled key: two recordings can still be recognised as the same speaker by whoever holds it. Voice transformation changes the acoustic identity cues and keeps selected content. Feature release shares embeddings or derived features that may still permit linkage or inference. Each gives up something different in exchange for what it protects.

Which of the three is right is not a question about the audio. Each has to be defended against a different attacker. Until someone says who that attacker is and what they already know, none of the three can be shown to work at all. The field has a standing answer to that question. The VoicePrivacy Challenge scores anonymisation against adversaries of declared knowledge — ignorant, lazy-informed and semi-informed — rather than against a hope. Run the same systems past all three and the numbers move. They move a long way.

Skipping that question costs something, and the VoicePrivacy 2020 Challenge printed the bill. The submitted anonymisation systems were scored more than once, against attack models of stated knowledge. Between those scorings the audio did not change at all. Only the attacker did.

The spread is the whole lesson. Against the ignorant attacker, many systems reached equal error rates above 50%. Against the lazy-informed attacker, the best results were 33–43%. Against the semi-informed attacker, the same systems came in at only 16–26%. That attacker's automatic speaker verification model had been retrained on anonymised speech. A team reporting the first number and a team reporting the third are describing identical audio.

The organisers stated the consequence plainly when they published the results in 2022: “Thus, assessing the performance of anonymization systems using an ASV system trained on original data leads to a false impression of protection.” False impression is the operative phrase. Nothing had gone wrong with the transforms. The protection reported at the top of the range was manufactured by the test, not by the audio.

Why does the transform not settle it by itself? Because identity in speech is not kept in one place. Speech is linkable through vocal, linguistic, behavioral and contextual cues. Voice de-identification changes selected identity evidence while attempting to preserve content or utility. Selected is the load-bearing word. What a method leaves alone is exactly what a retrained adversary works with.

So the privacy that results is never a property the transform can hand over. The 2020 systems were fixed objects. Their measured protection ran from above 50% down to 16–26% because the knowledge on the other side changed. Privacy depends on what the attacker knows, what auxiliary data exists, how often the same speakers are released, and what the downstream task is. That is why the challenge attaches a named adversary to every number it reports. It is also why its organisers bothered to rank their own attackers and call the last one the strongest known to date.

There is a second boundary, and it runs the other way. Perfect unlinkability may conflict with intelligibility, speaker consistency, emotion, pathology or forensic utility — the properties the audio was kept for in the first place. That is the trade word error rate sits in the score to measure, beside equal error rate. Defensible practice therefore ties one end to the other: define the protected unit at the start, evaluate privacy and utility at the end, with a named owner at each end.

Naming the unit and naming the adversary both depend on four words that are routinely swapped for one another. Swap one for another and you quietly change what evidence is owed, what unit it is measured in, and who holds the decision. The four have to be kept apart on purpose.

Those four words belong to different stages of the same job, and the job goes wrong when the stages arrive as one number. Privacy work fails quietly when measurement, modeling, decision and verification are collapsed into a single score. The score reads well. Nobody can say which stage produced it — or, as in the 2020 results, which attacker produced it.

The path below pulls those stages apart and takes them one at a time, in the order the challenge implies. Say who the attacker is and what unit is being protected before choosing the transform, not after reporting on it. The order matters because the 2022 plan shows what a late correction costs. The attacker was rebuilt, and every number measured against the old one stopped meaning what it had meant.

Where that trade gets settled depends on what the audio was kept for. Four settings differ enough that no single ruling covers them. Research corpora, call analytics, voice authentication and synthetic voices each leak identity in their own way, and each is counted in its own unit. Nothing that closed the 2020 evaluation closes any of them.

Across all four settings the report has the same shape. Closed- and open-set re-identification is only informative when the attacker is the strongest one the team can build. And the strongest one available is a moving figure, not a fixed setting.

The First VoicePrivacy Attacker Challenge made that concrete in 2025. Participants built attacker automatic speaker verification systems against anonymisation systems already submitted to VoicePrivacy 2024. In the authors' words, “The best attacker systems reduced the equal error rate (EER) by 25-44% relative w.r.t. the baseline.” The speech had not changed. The anonymisation systems had not changed. Published privacy figures fell by a quarter to nearly a half because other people were invited to attack them.

So report the unit the measure is in. Then say whose voices were tested, how uncertain the result is, and which attacker it was tested against, beside linkability across sessions and releases. A privacy figure with no adversary named beside it is not yet a privacy figure. One measured against last year's adversary has an expiry date.

A threat model is only worth writing if it names the adversaries the authors would rather not think about. That is the difference between a published ladder and a team's own reassurance. Because the ladder states what each attacker knows, another team can use it to challenge a privacy claim that was only ever tested at the bottom of it.

Start by writing down what defining the protected unit assumes: which of the waveform, transcript, embedding, metadata, timestamps and model outputs is in scope, and how far the release travels. Then give one counterexample to those assumptions — an adversary who has retrained on your anonymised output, or a second release of the same speakers. Then say what evaluating privacy and utility does with that counterexample: which number it moves, in which unit, and against which named adversary it was measured.

The last step, deletion and revocation, is the one that gets written as a sentence and tested as an operation. After a Big Brother Watch complaint, the UK Information Commissioner's Office moved to force HM Revenue & Customs to delete Voice ID biometric enrolments collected without explicit consent. A preliminary enforcement notice came on 4 April 2019, a final one on 9 May 2019. HMRC's chief executive, Sir Jonathan Thompson, wrote in a public letter dated 3 May 2019: “I have confirmed that HMRC will only retain Voice ID enrolments where we hold explicit consent.” Behind that sentence is arithmetic. Around 1.5 million customers had enrolled since the October 2018 changes, and those enrolments were kept. The roughly 5 million records without explicit consent — enrolments taken before October 2018 — were already being deleted, and the work was to finish well before the ICO's 5 June 2019 deadline. Five million voiceprints, one deadline, one letter naming who signed for it. That is what a finished revocation clause looks like when someone makes you execute it.