The Pulse
NVIDIA Releases 100M-Parameter Nemotron 3 Diarization Model
NVIDIA has released Nemotron 3 Diarization, an open-weight model that identifies when up to eight speakers talk, including overlapping speech. The model ranks first on VoiceArena’s initial Diarization-Bench with a 14.72% diarization error r
AI.info Team ·
A 100-million-parameter model from NVIDIA records a 14.72% diarization error rate on VoiceArena’s initial benchmark, placing it first among 12 systems and 17 configurations tested across roughly 22 hours of English conversations.
NVIDIA released Nemotron 3 Diarization on September 23, 2026, positioning it as an open-weight system for real-time and recorded conversations. The model identifies when speakers are active, preserves speaker channels across streaming audio and marks overlapping speech rather than forcing simultaneous voices into a single label.
Diarization is separate from speech recognition: the model produces speaker activity and timestamps, not a transcript. Developers can combine its output with an automatic speech recognition system to build transcripts that connect words to anonymous labels such as speaker_0 and speaker_1.
Nemotron 3 beats NVIDIA’s four-speaker baseline
VoiceArena’s initial Diarization-Bench places Nemotron 3 Diarization ahead of the next-ranked system, which recorded a 19.3% error rate under the benchmark’s zero-second collar setting. NVIDIA describes the result as an approximately 24% relative reduction, while cautioning that VoiceArena’s version-one evaluation and statistical analysis are not yet complete.
The benchmark covers 139 English-language conversations with overlapping speech included in scoring. NVIDIA says Nemotron 3 also ranked first under 100-millisecond and 250-millisecond collar settings, as well as across in-person and online recordings.
The model expands the company’s earlier Streaming Sortformer approach from four supported speakers to eight. It orders output channels by arrival time: the first newly detected voice becomes the first channel, the next new voice becomes the second, and so on. That arrangement helps keep anonymous speaker labels stable as new chunks arrive.
Eight channels, four latency settings
Nemotron 3 Diarization accepts 16 kHz, single-channel audio and converts it into Mel-spectrogram features before processing them with a 31-layer Transformer encoder. Its default output is a time-by-eight tensor of speaker-activity probabilities, allowing two or more channels to remain active in the same frame when people talk over one another.
NVIDIA recommends four operating points: 30.4 seconds for offline-style processing, 1.04 seconds for low latency, 0.64 seconds for very low latency and 0.32 seconds for ultra-low latency. Those figures describe input-buffer duration only; they exclude model computation, network transport, speech recognition and application processing.
The company says the model can technically operate with an 80-millisecond input buffer, but lists 0.32 seconds as the shortest recommended configuration. Shorter buffers reduce waiting time while generally lowering accuracy and throughput, leaving developers to select a setting based on the complete application pipeline.
Performance improves most with larger groups
Across eight evaluation conditions at 1.04-second input-buffer latency, NVIDIA reports lower diarization error for Nemotron 3 than for its previous four-speaker streaming model. The relative reductions range from 9.0% on CALLHOME-Part2 to 65.2% on the NOTSOFAR1 MHM condition, with an unweighted mean reduction of 41.0% across the datasets.
The results are not uniform in every subset. On the two-speaker CALLHOME subset, Nemotron 3 records a 5.98% error rate compared with 5.68% for the earlier baseline. Across the full CALLHOME-Part2 evaluation, however, the newer model improves the result from 10.32% to 9.10%, with larger gains in recordings containing more speakers.
On an NVIDIA RTX PRO 5000 using BF16 and batch size 32 with torch.compile(), the model reaches 15,113 times real-time factor at the 30.4-second configuration, compared with 2,619 times for the prior baseline. At 1.04 seconds, NVIDIA reports 865 times real-time factor against 136 times for the earlier model. Those are batched throughput measurements, not single-stream end-to-end latency.
Speaker labels are not identities
Nemotron 3 Diarization produces anonymous channels and does not determine that speaker_2 is a particular person. Applications may associate those channels with meeting metadata, user profiles or separate speaker-verification systems, but that identity layer sits outside the model.
NVIDIA trained the model with public and licensed speech data, including multispeaker recordings licensed from David AI. The company says additional David AI audio supplied simulated English and multilingual mixtures spanning 21 languages, reducing compound diarization error by 0.77 percentage points in its internal comparison, from 11.19% to 10.42%.
The release includes examples for NVIDIA NeMo Speech and supports WAV, FLAC, Opus and MP3 inputs on Linux systems with supported NVIDIA Ampere, Hopper or Blackwell GPUs. NVIDIA also lists integrations with Argmax Pro SDK 3, Baseten and DigitalOcean. The model is governed by the OpenMDW License Agreement, version 1.1, and the announcement warns that noise, reverberation, far-field microphones, long conversations and recordings with more than eight speakers can increase errors.