# NVIDIA details Dynamo tools for reinforcement-learning rollouts

Published 2026-11-19 · AI.info Team · AI.info

> NVIDIA describes how Dynamo routes reinforcement-learning rollouts, distributes updated model weights and handles worker failures. The company also reports customer performance figures for ModelExpress, including weight updates up to 15 times faster at CoreWeave.

[NVIDIA](https://developer.nvidia.com/blog/reducing-rollout-bottlenecks-in-reinforcement-learning-with-nvidia-dynamo/) says its Dynamo serving layer can reduce delays in reinforcement-learning training by routing model requests, distributing updated weights and recovering from worker failures. In a Nov. 19 technical post, the company described how the tools handle the inference work used to generate training samples and reported customer performance figures for its ModelExpress distribution component.

## Why rollouts slow training

In reinforcement learning, workers generate rollouts: sequences of model outputs, actions and interactions with an environment. The training framework scores those results, updates model weights and sends the weights back to workers for another round.

Rollouts differ by task, which makes the workload uneven. A reasoning rollout may consist of one response, while a coding-agent rollout can include exploring a repository, making edits, calling tools and running tests.

When a training step needs a group of samples, the slowest required rollout can hold up the next step. Distributing an equal number of requests across GPUs does not necessarily distribute the work evenly.

Repeated context is another source of work: attempts at the same task can begin with the same instructions, task description or repository context. Workers also need updated model weights in accordance with the training algorithm's requirements for sampling and acceptable staleness.

## Routing and weight updates

Dynamo sits between the software coordinating rollouts and the inference workers that generate them. The reinforcement-learning framework submits requests, Dynamo routes them to eligible capacity, and inference engines run the model.

The framework remains responsible for optimization, rewards, environments, checkpoints and deciding which samples are accepted. Dynamo handles routing, worker management, serving controls and telemetry, while ModelExpress distributes model weights and artifacts through supported integrations.

Requests go through a shared serving frontend, but weight refresh operations target selected workers under the framework's control. The framework determines when workers can generate samples and whether the results meet its training requirements.

Dynamo's router balances reuse of the KV cache against worker load, according to NVIDIA. Sending requests to a worker that has cached a shared prompt can avoid processing it again, but sending all requests to that worker can create a queue.

ModelExpress reuses weights already in the cluster instead of loading them from remote storage. Compatible workers can copy weights directly from another GPU over NIXL. For subsequent updates, delta transfer sends a compressed representation of the changes rather than the full model; workers reconstruct the new weights from the matching base version.

## Failures and reported results

Dynamo detects a lost worker, directs new requests to eligible workers and adds replacements to routing when they are ready. Supported configurations also allow migration of eligible requests already in flight and graceful draining. The training framework handles interrupted samples and checks that replacement workers have the required weights.

A replacement worker can load a checkpoint and apply subsequent deltas to catch up with the training run. Healthy workers can continue generating rollouts while capacity is restored.

NVIDIA says [CoreWeave](https://ai.info/companies/coreweave) uses Dynamo and ModelExpress to speed up model weight updates by up to 15 times. The company also reports that H Company used them to reduce average training-step time by up to 20% and GPU-hours by up to 5%.

Open-source integrations connect Dynamo with the verl, NeMo RL, Slime and prime-rl reinforcement-learning stacks. The post also identifies [Microsoft](https://ai.info/companies/microsoft) AI, Cognition and Prime Intellect as users of Dynamo in reinforcement-learning workflows.

## Sources

1. Original source · Blog post [NVIDIA Technical Blog](https://developer.nvidia.com/blog/reducing-rollout-bottlenecks-in-reinforcement-learning-with-nvidia-dynamo/) developer.nvidia.com

## Companies in this story

- [Nvidia](https://ai.info/companies/nvidia)

## Related stories

- [Google details bitwise-matched RL training across 64 TPU chips](https://ai.info/news/google-bitwise-matched-rl-training-tpu)
- [NVIDIA puts four-node DGX Spark developer kit on sale](https://ai.info/news/nvidia-four-node-dgx-spark-hpc-developer-kit)
- [Nvidia’s private-company stakes reach $65.5 billion, TechCrunch reports](https://ai.info/news/nvidia-private-startup-stakes-65-billion)
- [NVIDIA reports Vera Rubin gains in MLPerf Training debut](https://ai.info/news/nvidia-vera-rubin-mlperf-training-endpoints-results)
- [RadixArk releases DaisyRose-27B with post-training efficiency claims](https://ai.info/news/radixark-daisyrose-27b-post-training-efficiency)

## Explore

[More articles](https://ai.info/news)

---

Canonical: https://ai.info/news/nvidia-dynamo-reinforcement-learning-rollout-bottlenecks
