Skip to content
AI.info

The Pulse

Kyutai releases AURA-5B for non-commercial talking-head video

AURA-5B turns a portrait and speech audio into a talking-head video, with weights available for non-commercial research. Kyutai reports that the model generates 720p video at 9.7 frames per second on one H100 GPU, below its playback rate.

Kyutai releases AURA-5B for non-commercial talking-head video

Kyutai has made AURA-5B available to turn a single portrait and speech audio into a talking-head video, with model weights restricted to non-commercial research. In an optimized pipeline, the model produces 720p video at 9.7 frames per second on one H100 GPU, according to its model card. That is below the 25 Hz playback rate used by the model.

A preprint describing the work was posted to arXiv on November 19, 2026. It also presents a smaller model, AURA 1.3B, which the researchers report can generate lower-resolution video in real time.

Streaming in chunks

AURA-5B generates video a chunk at a time rather than waiting for a longer stretch of audio. It needs the audio for the current chunk, without the multi-second audio look-ahead used by the bidirectional teacher model.

Each chunk represents about 0.64 seconds and contains four latent frames. A rolling cache keeps the first chunk as an identity anchor and retains the most recent chunk for context. Kyutai says this keeps memory use constant and allows streams of 60 seconds or longer.

The model is a four-step causal student distilled from TalkVerse, a bidirectional teacher that uses 50 denoising steps. The team reports that AURA-5B is 17 times faster than its teacher end-to-end and 30 times faster on the diffusion transformer alone.

Training and available files

The model card lists Snap's TalkVerse-S2V-5B as the base model, itself built on Wan2.2-TI2V-5B. Training used two stages: the first fitted the student to the teacher's trajectories using 30,000 synthetic clips; the second used distribution matching distillation with backward simulation, training on the student's own rollout states.

Both stages fine-tuned LoRA adapters on 64 H100 GPUs. Kyutai provides a merged standalone checkpoint with 5.7 billion parameters and a 22.8 GB file size. It also provides LoRA-only checkpoints that must be applied to the TalkVerse base, along with a standalone checkpoint from the first training stage.

The reference generation code downloads the final merged model and separate text, video and audio components. With compilation enabled, that code runs at about 8 frames per second on one H100, according to the model card; the higher reported rate comes from an optimized pipeline.

Limits and licensing

On the HALLO3 test, the researchers report a Sync-C lip-synchronization score of 4.30 for AURA-5B, compared with 5.06 for Hallo3 and 5.42 for Wan-S2V. The model card says the model was trained and evaluated on talking-head videos driven by English speech; other languages, non-speech audio and non-portrait framings have not been tested.

All AURA-5B checkpoints are derived from the Snap model and carry its non-commercial license for research use. The separately downloaded components retain their own licenses. Kyutai also cautions against impersonation, non-consensual content and disinformation, and calls for consent from portrait subjects and disclosure that videos are AI-generated.

The preprint also describes AURA 1.3B, which generates 576p video. The researchers report a rate of 20.7 frames per second on a single GPU for a 16 Hz playback rate.

Sources

  1. Original source Kyutai huggingface.co

Other pages read

  1. arXiv arxiv.org

Related stories

Explore

More articles