# NVIDIA's Reshard Refit Achieves a 2.7x to 5.7x Speedup Across Various NeMo RL Benchmarks

Published 2026-11-06 · AI.info Team · AI.info

> NVIDIA's Reshard approach transfers only the theoretical minimum amount of model-weight data between training and inference GPUs, the company says. In NeMo RL benchmarks, Reshard refit achieved a 2.7x to 5.7x speedup over conventional gather-and-broadcast refit.

[NVIDIA](https://developer.nvidia.com/blog/accelerating-reinforcement-learning-weight-refit-with-the-reshard-collective/)'s Reshard-based reinforcement learning weight refit achieved a 2.7x to 5.7x speedup over conventional gather-and-broadcast refit across various NeMo RL benchmarks, the company says. The benchmarks covered NVIDIA DGX H100 and DGX GB200 systems. In a separate refit-only microbenchmark, Reshard refit took 4.3 seconds for the Qwen3.8 2.4T BF16 model.

## Existing gather-and-broadcast refit implementations commonly reconstruct complete parameters

During reinforcement learning post-training, training trajectories are generated on the fly by running inference with the current policy model. In a disaggregated setup, inference and training run in parallel on separate GPU resources. The training GPUs update the policy, while the inference GPUs periodically receive the latest weights; NVIDIA calls that transfer refit.

Training and inference can distribute weights differently because they have different computational requirements and parallel mappings. Existing refit implementations commonly gather local shards on the training GPUs to reconstruct a complete parameter, broadcast that tensor to inference GPUs, then keep the shard each inference GPU needs. The full-tensor intermediate creates redundant communication in both the gathering and broadcasting stages, NVIDIA says.

Kimi K3's 2.8 trillion parameters correspond to approximately 5.6 TB of BF16 weights, NVIDIA estimates, while Qwen 3.8 2.4T A95B's 2.4 trillion parameters correspond to approximately 4.8 TB. Counting the all-gather and broadcast stages, the estimated logical payload is approximately 11.2 TB for Kimi K3 and 9.6 TB for Qwen 3.8 2.4T A95B. The payload for today's largest open weight models reaches around 10 TB per refit, the company says.

## Reshard maps source shards to destination shards

In NVIDIA's framing, refit is a transfer between two PyTorch distributed tensor (DTensor) layouts. Each layout describes the tensor's global shape, the logical arrangement of GPUs, and how the tensor is sharded or replicated across them. A Reshard operation takes source and destination DTensor descriptions and transfers the logical tensor from its source distribution into its destination distribution.

In NVIDIA's example, an 8-by-8 weight matrix is split by rows across two training GPUs and by columns across four inference GPUs. Each training GPU holds a 4-by-8 shard, and each inference GPU needs an 8-by-2 shard. Each destination can obtain its required shard by receiving a 4-by-2 region from each source, assembling its required shard without reconstructing the complete matrix.

NCCL M2N Reshard is an open-source reference implementation of the Reshard collective, available through the NVIDIA/nccl-extensions [GitHub](https://ai.info/companies/github) repo. It transfers a tensor distributed across M source GPUs into the layout required by N destination GPUs. In RL refit, the training GPUs form the source group and the inference GPUs form the destination group.

## NCCL M2N Reshard eliminates redundant network transfers with NVLink exchanges

NCCL M2N Reshard calculates the exact intersections between source and destination shards, then maps those transfers onto the physical GPU network topology. It groups destination replicas within NVIDIA NVLink domains and distributes incoming contributions among replica ranks. The receiving GPUs exchange their subsets over NVLink to assemble complete shards.

For a TP4-to-TP2 x DP2 example, direct point-to-point transfers send each source shard across the network to both data-parallel replicas, requiring eight network transfers, according to NVIDIA. Reshard sends each of the four source shards over the network once, then uses NVLink exchanges to complete the replicas. Network transfers and local exchanges are pipelined to hide each other's latency.

## NeMo RL applies Reshard to feed-forward projection weights

NVIDIA's reference implementation is provided in the NeMo RL framework. NeMo RL is an open source framework designed to streamline and scale RL post-training. The framework needs to provide DTensor descriptions for each parameter in the training and inference instances. The variety of layer types and sharding arrangements makes it challenging to cover every edge case through Reshard alone, NVIDIA says.

NVIDIA's NeMo RL design uses a hybrid path: feed-forward projection weights use Reshard, while all other small weights use conventional gather-and-broadcast. Feed-forward projection weights account for around 95% of parameters in the latest LLMs and dominate the refit payload, NVIDIA says. The hybrid design retains the same functional coverage as conventional gather-and-broadcast refit, the company says.

## NeMo RL benchmarks cover a range of models on NVIDIA DGX H100 and NVIDIA DGX GB200 systems

On NVIDIA DGX H100, Qwen 3 235B refit latency was 6.4 seconds with gather-and-broadcast and 1.9 seconds with Reshard. For DeepSeek v3 671B on the same platform, latency was 14.9 seconds with gather-and-broadcast and 2.6 seconds with Reshard. NVIDIA tested both models with different training-to-inference parallel layouts.

On NVIDIA DGX GB200, DeepSeek v3 671B refit latency was 6.1 seconds with gather-and-broadcast and 2.3 seconds with Reshard. Nemotron Ultra v3 on DGX GB200 had latency of 8.2 seconds with gather-and-broadcast and 2.3 seconds with Reshard. NVIDIA's NeMo RL refit guide and NCCL Reshard configuration requirements describe supported backends, layouts, and precision modes.

## Sources

1. Original source · Blog post [NVIDIA Technical Blog](https://developer.nvidia.com/blog/accelerating-reinforcement-learning-weight-refit-with-the-reshard-collective/) developer.nvidia.com

## Companies in this story

- [Nvidia](https://ai.info/companies/nvidia)

## Related stories

- [NVIDIA details Dynamo tools for reinforcement-learning rollouts](https://ai.info/news/nvidia-dynamo-reinforcement-learning-rollout-bottlenecks)
- [Google details bitwise-matched RL training across 64 TPU chips](https://ai.info/news/google-bitwise-matched-rl-training-tpu)
- [RadixArk releases DaisyRose-27B with post-training efficiency claims](https://ai.info/news/radixark-daisyrose-27b-post-training-efficiency)
- [NVIDIA’s 120B Nemotron 3.5 Super VL is available on SageMaker](https://ai.info/news/nvidia-nemotron-35-super-vl-sagemaker-jumpstart)
- [NVIDIA details 2B-parameter model for onboard AV reasoning](https://ai.info/news/nvidia-alpamayo-2-edge-onboard-av-reasoning)

## Explore

[More articles](https://ai.info/news)

---

Canonical: https://ai.info/news/nvidia-faster-rl-weight-refit-reshard
