The Pulse
ESPnet Releases YODAS v3 With 1.1 Million Speech Hours
YODAS v3 holds more than 1.1 million hours of 48 kHz, mostly multi-channel speech under a CC BY 3.0 license. Its paper and dataset card count languages two ways, and most of its transcripts come from automatic YouTube captions.
AI.info Team · Written with AI · fact-checked ·
The ESPnet team announced YODAS v3 on Sunday, a speech corpus of more than 1.1 million hours of audio released under a Creative Commons Attribution 3.0 license and hosted on Hugging Face. The release post says the collection triples the roughly 370,000 hours of the first YODAS, published almost three years ago, and it is the largest release in the series so far.
The accompanying paper, by William Chen, Shinnosuke Takamichi, Sayaka Shiota, Satoru Fukayama, Samuele Cornell and Shinji Watanabe, lists Carnegie Mellon University, Keio University, Tokyo Metropolitan University and Japan’s National Institute of Advanced Industrial Science and Technology as affiliations. Its authors call YODAS v3 the largest open speech dataset to date and the first truly large-scale speech corpus with high-fidelity stereo audio. A footnote adds that after the paper was accepted, the team removed duplicated videos, cutting the total from 1.4 million to 1.1 million hours.
48 kHz files, with the real bandwidth measured
The vast majority of open speech datasets, the paper notes, distribute mono audio at 16 kHz or 24 kHz. YODAS v3 keeps the format the audio was downloaded in: 48 kHz multi-channel OPUS files. The release post argues that makes the data usable for speech synthesis, enhancement and audio codecs as well as speech recognition.
A 48 kHz file header proves little on its own. The Opus codec always decodes to 48 kHz whatever bandwidth it actually encoded, and recordings may come from laptop microphones or USB headsets with limited bandwidth. So the team estimated each recording’s effective bandwidth from its frequency content. According to the paper, 67.3% of the data has an effective sampling rate of 44.1 kHz and 92.5% sits above 32 kHz. The estimate ships as metadata, so users can filter for the fidelity a task needs.
The authors ran a similar check on channels, subtracting each pair of channels and flagging those that were effectively identical. Over 71% of the audio files, about 780,000 hours, are truly multi-channel, the paper says, with the vast majority stereo. A small amount is effectively three or four channels, which the team traces to 5.1 surround-sound recordings.
Two ways of counting languages
The paper assigns each video a language from its YouTube-provided locale, which it uses as a proxy “due to the cost of automatic language ID at scale.” On that basis it reports 147 languages, with 22 languages above 10,000 hours and 73 above 5,000 hours, and a mean of 7,400 and a median of 5,180 hours per language.
The dataset card on Hugging Face describes two sources of language IDs: the language attached to a video’s transcript, and the uploader’s locale. The team wrote that it originally reported the locale-based distribution in the paper, but that the transcript-based IDs are “likely more accurate, if available.” The download is therefore organized by transcript language, with both IDs kept in the metadata and files lacking a transcript placed in an “unk” directory. The card and the release post describe coverage as more than 100 languages, 34 of them with more than 1,000 hours each.
Weak labels, mostly automatic
The paper calls YODAS v3 a weakly labeled corpus. According to the dataset card, approximately 75% of the data has transcripts and around 60% of the non-English data has English translations. All transcripts and translations carry sentence-level timestamps, and 95% of the transcripts also have word-level timestamps. The card warns that language IDs are assigned per file and may not be entirely accurate on individual sentences.
Human-made subtitles are scarce. Manual subtitles account for only 3.7% of the data, against 20% in YODAS v2, according to the paper; the other transcripts are automatic YouTube captions. The authors suggest better automatic subtitling has made manual subtitles less necessary. Their crawler built a separate keyword list for each language from its Wikipedia dump, searched only videos uploaded under a Creative Commons license, favored recent uploads and dropped any video already in YODAS v2, so the two releases can be combined.
What the baseline models showed
To test the transcripts, the team trained monolingual speech recognition models on a random subset of seven languages: 4,500 hours of English and roughly 100 hours for each of the other six. The models started from the 102-million-parameter OWSM v4 base and were trained for 40,000 steps with ESPnet, then scored on Common Voice. Error rates almost always improved as less data was filtered out. The authors wrote:
“the best models for each language (except for English) were trained without any score-based filtering, showing that YODAS v3 is directly usable out-of-the-box.”
William Chen, Shinji Watanabe and co-authors, YODAS v3 paper (Interspeech 2026)
That contrasts with YODAS v2, which the paper says needed significant filtering to avoid failed training runs with word error rates above 100. The team also trained DAC neural audio codecs on a random 1,200-hour subset at 16, 24 and 48 kHz. All of them scored lower on intelligibility on a YODAS v3 test set than on the clean LibriTTS set, which the authors attribute to background noise and multiple sound sources. According to the paper, the 48 kHz model trained on YODAS v3 outperformed every lower-sample-rate model and the models trained on AMUSE, a 33,000-hour mix of speech, music and sound.
The paper is part of Interspeech 2026, and the dataset card asks anyone who re-releases a large share of the audio in a new dataset, benchmark or repository to cite it.