TTS Evaluation, Consent, Provenance, and Deepfake Response

Text-to-speech quality is not one property that a system has more or less of. It is ten, and each has to be shown separately: content correctness, pronunciation, intelligibility, naturalness, speaker similarity, prosody, latency, accessibility, authorization, and provenance. Nothing holds them together. A system can improve on several of them while getting worse on the rest.

So the work splits three ways. Content evaluation, perceptual evaluation and governance evaluation are run by different people, for different reasons. All three have to be commissioned while the architecture can still change. Two of them ask questions that no amount of later listening will answer.

The rest of this lesson works from four places where that split was tested in public and the record survives. A $6,000,000 forfeiture order over a cloned voice that cost $150 to make. A listening test that gave identical 2013 recordings a different score in 2022. A state statute that put the word “voice” inside a property right. And the EU AI Act, which sets a date after which marking synthetic audio at generation stops being good practice.

A robocall went out to New Hampshire voters in President Biden's voice, telling them not to vote in the primary. The voice was a clone. It cost $150 to make. The forfeiture came to $6,000,000.

The order is FCC 24-104, adopted on 26 September 2024. Its first paragraph says what the recordings were: “Kramer's illegal robocalls carried a deepfake generative artificial intelligence (AI) voice message (Deepfake Message) that imitated U.S. President Joseph R. Biden, Jr.'s voice and encouraged potential voters not to vote in the then-upcoming Primary Election.” The audio was made with Eleven Labs software, the order records, and paid for with $150. Kramer is a political consultant.

Now ask what a listening panel would have contributed. A panel is asked how a voice sounds. Nothing the Commission objected to turns on that — not naturalness, not intelligibility, not prosody, not latency, not long-form stability. The order is about whose voice it was, what the people who answered the phone were told, and whether anybody had agreed to any of it. A five-point rating scale does not contain those three questions and cannot be extended to contain them. A high naturalness score on that audio would have been accurate and beside the point at the same moment.

The same gap opens in the quieter direction. A proper name mispronounced in a narration is a content error, and it does not surface in a naturalness rating either. The listener hears a fluent, pleasant delivery of the wrong word, and rates the delivery. In both directions the mechanism is identical. The instrument answers its own question well and stays silent on the rest.

Those instruments belong to a set of three. Content evaluation asks whether the intended words and entities were spoken correctly — text coverage, pronunciation, critical-entity error. A mispronounced name belongs there. Perceptual evaluation asks how the output sounds to listeners under a controlled protocol, and reports naturalness, intelligibility, similarity and prosody. That is the question P.800 and P.808 formalise, and the one the Blizzard replay showed to be a property of the sitting rather than of the system. Governance evaluation asks who was entitled to that voice, for what use, and what the file says about its own origin.

The third column is the one teams treat as internal policy preference. In Tennessee it stopped being one. The ELVIS Act took effect on 1 July 2024, adding “voice” to a property right that until then covered only name, photograph and likeness. The full name is the point of it: the Ensuring Likeness, Voice, and Image Security Act of 2024, in place of the state's Personal Rights Protection Act of 1984. The definition is written so that a synthesised voice cannot slip out of it: “‘Voice’ means a sound in a medium that is readily identifiable and attributable to a particular individual, regardless of whether the sound contains the actual voice or a simulation of the voice of the individual;”. The Act creates civil liability both for publishing an unauthorized voice and for distributing a tool whose primary purpose is producing one. The vendor of the cloning system stands in the same column as its user.

A strong result in one of the three says nothing about the other two. A panel is not run badly when it comes back clean on a voice nobody was entitled to use. It is simply the wrong instrument for two thirds of the question, and it returns a confident number either way.

Ten quality dimensions cannot be argued about inside one score, so the order of the work carries as much weight as its content. Four stages stay separable, each checked on its own terms. Define the use and harm model — content, voice, listeners, channel, languages, consequences. Build objective checks — text coverage, pronunciation, critical tokens, duration, technical artifacts. Run controlled listening, with naturalness, intelligibility, similarity, style and preference asked as separate questions, under the protocol P.800 or P.808 specifies. Keep the Blizzard result beside it, a standing reminder that the number belongs to the sitting. Then test governance — consent, provenance, generation logs, abuse controls, revocation, incident response.

The last stage now has a deadline attached to it. Synthetic audio will have to carry a mark a machine can read. Article 50(2) of the EU AI Act: “Providers of AI systems, including general-purpose AI systems, generating synthetic audio, image, video or text content, shall ensure that the outputs of the AI system are marked in a machine-readable format and detectable as artificially generated or manipulated.” Article 50(4) adds a disclosure duty on deployers of deep-fake audio. Both apply from 2 August 2026. Machine-readable marking at generation — the AudioSeal design point — is on a calendar, not on a wish list.

Coming last gives governance testing a limit worth stating plainly. It compares the system against the use and harm model that somebody wrote down. Whatever was never written down has nothing to fail against. The release goes out with a clean sheet.

Four terms circulate through these reviews as if they were grades on the same report card. They are not. They belong to three separate governance conversations, and reading them as one quality score is how a release passes on the strength of the wrong evidence.

A listening score that hides content errors is exposed by one thing only: measuring the content. Text coverage, pronunciation and critical-entity error are where a mispronounced name stops being an impression and becomes a number. In a narration a proper name is a critical entity by definition.

Report every figure with its unit. Report the perceptual figures with the conditions that produced them — who listened, how many of them were analysed, how uncertain the score is, and what they heard it on. The same stimuli moved from 3.81 to 2.62 when the surrounding systems changed. And report the governance figures at the same time as the rest. Those were the only ones with $6,000,000 attached to them.

A scorecard has one job: to let another team see, from the record alone, whether a pleasant-sounding voice was carrying content errors or an authorization it never had. Keep it to three things. Write down what your use and harm model assumes. Write down one case that breaks that assumption. Then say what the testing and governance steps do about that case.

Four steps make it operational. First, separate the dimensions and assign independent acceptance criteria to content, perception and governance. Second, sample difficult material — names, numbers, long text, emotion, silence, code-switching. Third, blind the listening test: randomize systems, control level, report listener uncertainty, and record the population, since that population is part of the result. Fourth, run an abuse drill — unauthorized cloning, viral redistribution, takedown, evidence preservation.

The fourth step has a published baseline for how low the bar sits. Consumer Reports assessed six voice-cloning products and published the result on 10 March 2025: Descript, ElevenLabs, Lovo, PlayHT (also branded PlayAI), Resemble AI and Speechify. Only Descript and Resemble AI took technical steps to prevent unauthorized cloning. Of the other four the report says: “Rather, these companies require only that users check a box confirming they have the legal right to clone the voice or make a similar self-attestation.” Even Descript's required spoken consent statement could be defeated by generating that statement with a different voice-cloning service. So the drill is not hypothetical. Ask the same question of the consent artifact in your own pipeline. Is it evidence, or a tickbox that any other vendor's product can produce on demand?