The Pulse
NVIDIA releases Sigma diffusion language models in two sizes
Nemotron-Labs-Sigma generates text in blocks by denoising continuous token representations. NVIDIA reports a 3.46-fold average decoding speedup for its 8B model in tests on a GB200.

NVIDIA has released a family of language models that generate text in blocks by denoising continuous representations of tokens, with versions at 3 billion and 8 billion parameters. The models, called Nemotron-Labs-Sigma, are designed for reasoning and coding, according to NVIDIA Research.
MarkTechPost reports that the collection contains 14 open checkpoints, including base, instruction-tuned, distilled and self-speculative variants. The 8B base model's card lists its Hugging Face release date as November 12, 2026.
Generating text in blocks
Sigma begins each block with Gaussian noise and repeatedly denoises a representation conditioned on the preceding text. Once the block has been refined, the model reads out discrete tokens and moves to the next block.
The representation uses 16-dimensional token embeddings, learned together with the denoising network. Training starts from pretrained autoregressive weights and uses a shared backbone for an autoregressive stream processing clean tokens and a diffusion stream denoising their embeddings.
During generation, the model can adjust the trajectory using classifier-free guidance and score temperature. It also feeds the previous conditional prediction into the next denoising step, a technique the project calls self-conditioning.
Fewer steps and speculative decoding
NVIDIA says distilled models generate eight tokens per forward pass on math tasks and four on coding, with negligible accuracy loss and preserved diversity. Distribution matching is used to distill the model for generation with fewer denoising steps.
In a separate method, the distilled diffusion model drafts blocks that its own autoregressive stream verifies. NVIDIA says this self-speculative decoding reproduces the autoregressive model's greedy output.
On an NVIDIA GB200, the 8B model achieved an average throughput speedup of 3.46 times over its autoregressive decoding, according to the research project. That figure describes the tested self-speculative method rather than a general speedup for every use of Sigma.
Results and availability
In NVIDIA's evaluations, the 8B base model averaged 71.16 across NVIDIA's six-task suite, ahead of NLD-8B's 70.22, MarkTechPost reports. The same report says Sigma-8B scored 65.12 on HumanEval, against 62.80 for the comparison model, but scored 65.74 on MBPP against 67.00.
The 8B base model's maximum context is 4,096 tokens, including the prompt and generated text. Its model card describes the base version as a starting point for research and fine-tuning rather than an instruction-following assistant; a corresponding instruct variant is available.
The 8B base model runs on a CUDA-capable NVIDIA GPU using bfloat16, and CPU inference is not supported. Its use is governed by the NVIDIA Nemotron Open Model License, and the model card says it is ready for commercial or non-commercial use.
Sources
- Original source NVIDIA Research research.nvidia.com
Other pages read
- NVIDIA on Hugging Face huggingface.co
- MarkTechPost marktechpost.com
Companies in this story
Related stories
- NVIDIA details two-prompt workflow for physical AI training data
- NVIDIA posts draft models for Qwen3.8-27B decoding
- METR finds AI models usually match or beat skilled humans on its coding tasks, with time horizons doubling every 3.6 months
- OpenAI says agents completed 46% of delegated two-to-four-hour tasks
- NVIDIA details sparse-attention trade-offs for long-context AI