tools
Moshi
Moshi is an open-source speech-to-speech model for real-time, full-duplex voice conversations.
Moshi listens and speaks directly through audio streams instead of using separate speech recognition, text generation, and text-to-speech stages. It models the user’s and assistant’s audio in parallel and produces an inner-monologue text stream to guide responses.
Researchers and developers can run it locally through PyTorch, MLX, or Rust backends, use its web interface, or adapt the code for experiments. It speaks English and requires substantial hardware for some configurations; Windows is not officially supported.
Features
- Full-duplex spoken dialogue with simultaneous listening and speaking
- Direct speech-to-speech generation with parallel audio streams
- Inner-monologue text stream to improve response generation
- Streaming Mimi neural audio codec for 24 kHz audio
- PyTorch, MLX, and Rust/Candle inference backends
- Local web interface and command-line client
- Three released model variants, including Moshika and Moshiko
- Code under MIT and Apache-2.0 licenses; model weights under CC-BY 4.0
Use cases
- Build low-latency voice conversation prototypes
- Research speech-native language models
- Run voice dialogue locally on Mac or Linux hardware
- Fine-tune voice dialogue behavior for experiments
- Create custom web or command-line speech interfaces