# Up to 70% More Audio per Dollar on Trainium2, Deepgram Reports

Published 2026-10-19 · AI.info Team · AI.info

> A tuned version of Deepgram’s Nova-3 speech-to-text model processed 11% to 70% more audio per dollar on Trainium2, Deepgram and AWS say. The result applies to offline batch transcription on one EC2 instance, not to every Deepgram workload.

## A 70% ceiling, with a narrow scope

A tuned version of Deepgram’s Nova-3 speech-to-text model processed 11% to 70% more audio per dollar on Trainium2 than the compute options Deepgram had been using, AWS and Deepgram report. The upper figure is not a general result for all speech recognition: the comparison covers offline, throughput-oriented batch workloads, including contact-center analytics and medical transcription. The metric was calculated using instance prices at the time of evaluation, AWS says.

Source [aws.amazon.com](https://aws.amazon.com/blogs/machine-learning/how-deepgram-optimized-voice-ai-at-scale-with-aws-trainium/)

That scope matters. The post describes a cost-efficiency result for batch inference, not a claim that Trainium2 improves latency or cost across Deepgram’s real-time services, text-to-speech models or voice agents. It also does not identify the comparison instances or provide absolute audio-throughput figures, leaving readers without enough detail to reproduce the full price-performance comparison.

## Eight Nova-3 replicas on one instance

The final system runs eight copies of Nova-3 on a single Amazon EC2 trn2.3xlarge instance. Deepgram and AWS spent several months porting the model from standard PyTorch code to Trainium2 and tuning it for the chip. The goal was to make more efficient use of accelerator time on workloads that process large batches of recordings.

The team used TorchNeuron, AWS’s PyTorch integration for its Neuron compiler, and compiled the encoder and decoder as separate static-shape graphs. The compiled model ran every operation on Trainium2 without detected CPU fallbacks, AWS says. The team also checked layer-level outputs against CPU results and monitored word error rate against Deepgram’s reference values.

## Kernel tuning and a Rust serving path

After establishing a working baseline, the engineers replaced selected operations with fused kernels from the Neuron Kernel Interface Library, then used profiling to identify further bottlenecks. They wrote custom kernels for feed-forward layers, depthwise one-dimensional convolution and two-dimensional convolution. The custom feed-forward kernel cut latency in half versus its reference kernel at batch size 96, while the depthwise-convolution kernel was 1.5 times faster at that batch size, AWS reports.

A separate engineering task was fitting the model into Deepgram’s production serving stack, which is written in Rust. The team built a runtime in which Rust coordinates model replicas through a shared work-stealing queue, while a C layer loads compiled Neuron executable files through AWS’s Neuron Runtime API. The runtime reproduced the reference token sequences at every tested batch size, AWS says.

Against a multiprocessing Python baseline, the Rust runtime produced a 7.5% throughput gain at the batch size that maximized throughput. The post says the advantage is larger at low batch sizes, where repeated decoder calls make host-side dispatch more significant. That runtime result is distinct from the headline cost-per-dollar range: it measures throughput against a Python serving baseline, not the 11% to 70% comparison against Deepgram’s prior compute options.

## Self-hosting is the next stated use

The efficiency gains could also benefit customers running Deepgram models in their own virtual private clouds through Amazon SageMaker AI, AWS says. Deepgram and AWS are continuing work on setup, reliability and observability for that deployment path, but the post does not publish a separate benchmark for customer-hosted systems.

For now, the concrete result is a tuned Nova-3 batch-inference configuration on one trn2.3xlarge instance, with eight model replicas and a reported 11% to 70% increase in audio processed per dollar against Deepgram’s previous compute options. The measured range depends on the batch workload and the instance pricing used during the evaluation.

## Sources

1. Original source · Blog post [AWS Machine Learning Blog](https://aws.amazon.com/blogs/machine-learning/how-deepgram-optimized-voice-ai-at-scale-with-aws-trainium/) aws.amazon.com

## Companies in this story

- [Amazon](https://ai.info/companies/amazon)
- [Amazon Web Services](https://ai.info/companies/amazon-web-services)

## Related stories

- [Deepgram to Acquire Daily and Keep Pipecat Open](https://ai.info/news/deepgram-acquires-daily-pipecat)
- [Apple Researchers Find Telling Languages Apart Helps Speech Models](https://ai.info/news/apple-language-separation-multilingual-speech-models)
- [AWS Adds Usage-Based Billing for SageMaker Streaming Models](https://ai.info/news/aws-sagemaker-streaming-usage-based-billing)
- [Apple Finds Online Teachers Improve Federated Speech Recognition](https://ai.info/news/apple-online-teachers-federated-speech-recognition)
- [FuriosaAI Raises $620 Million to Expand RNGD Chip Production](https://ai.info/news/furiosaai-620-million-series-d-rngd-production)

## Explore

[More articles](https://ai.info/news)

---

Canonical: https://ai.info/news/deepgram-trainium2-audio-per-dollar
