The Pulse
Kyutai’s Voice of Reason Scores 70.6% on GSM8K
Kyutai’s GLM-4-Voice of Reason is a speech-to-speech model that reasons before speaking and scores 0.706 on a 1,310-item written-channel GSM8K evaluation.
AI.info Team ·
Kyutai’s GLM-4-Voice of Reason is a speech-to-speech model trained to reason before it speaks. Its model card reports a score of 0.706, or 70.6%, on a 1,310-item GSM8K test evaluation using the written channel and gpt-4o-2024-11-20 as the judge.
The model is built on GLM-4-Voice-9B and is trained with reinforcement learning against a binary language-model judge on mathematical word problems. Kyutai’s model card says there is no supervised fine-tuning stage for the reasoning behavior: the model learns that behavior from the reward signal.
Reasoning is part of the spoken response
GLM-4-Voice of Reason accepts spoken input and produces a spoken answer. Its reasoning is not placed in a separate hidden channel. Instead, the model works through the problem in the answer it speaks, step by step.
The model card’s example instructs the system to respond in an interleaved manner, with 13 text tokens followed by 26 audio tokens. In the demonstrated workflow, the generated output contains both text tokens and speech tokens. The text can be decoded as the complete response, while the speech tokens represent the audio codes that will be converted into a waveform.
That design keeps the reasoning and answer inside the same speech-generation process. The model card describes the system as a speech-to-speech model rather than a conventional voice stack that first transcribes speech, sends the text to a separate language model and then synthesizes the result.
A benchmark result tied to a specific setup
The reported 0.706 GSM8K score comes with several qualifications. It covers 1,310 test items, uses the written channel and relies on gpt-4o-2024-11-20 as the evaluating judge. The figure therefore describes a particular evaluation protocol rather than a general measure of mathematical ability across every spoken setting.
The model card frames the task as mathematical word-problem solving. Because the judge supplies a binary language-model assessment, the result reflects both the model’s generated response and the evaluation procedure used to classify it. The card does not present the score as a symbolic-verifier result.
Local deployment requires the GLM-4-Voice components
Kyutai’s release is intended to run locally. The model card says the audio front end is inherited from GLM-4-Voice and is not included in the repository. Users must clone the GLM-4-Voice repository to obtain the speech tokenizer, then use the model’s generated speech tokens with the GLM-4-Voice decoder to produce a waveform.
The published example uses Transformers, PyTorch, Torchaudio, Accelerate, Tiktoken and SoundFile. It accepts an audio file that Torchaudio can read, including files with any sample rate supported by the audio library. The example loads the checkpoint in bfloat16 and places it on a CUDA device.
Kyutai says the printed script was run on a single H100 GPU with Transformers 4.47.1 and PyTorch 2.8.0. The model card also warns that the appropriate PyTorch build depends on the system’s CUDA driver, since newer wheels may require a newer CUDA version than some clusters provide.
An open checkpoint for spoken reasoning research
GLM-4-Voice of Reason gives researchers an openly available checkpoint for studying reasoning during audio generation. Its design focuses on having one speech-native model process an audio question, generate intermediate reasoning and deliver the answer as speech.
The release is therefore useful both as a model and as a reproducible local experiment. Its benchmark result is 70.6% on the specified GSM8K setup, while the accompanying code shows how to extract speech tokens, decode the textual response and pass the audio codes to a decoder. The model card does not establish broader performance claims beyond that evaluation.