The Pulse
NVIDIA's Reshard Refit Achieves a 2.7x to 5.7x Speedup Across Various NeMo RL Benchmarks
NVIDIA says its Reshard approach transfers only the theoretical minimum amount of model-weight data between training and inference GPUs. In NeMo RL benchmarks, Reshard refit achieved a 2.7x to 5.7x speedup over conventional gather-and-broad
AI.info Team · Written with AI · fact-checked ·
NVIDIA says its Reshard-based reinforcement learning weight refit achieved a 2.7x to 5.7x speedup over conventional gather-and-broadcast refit across various NeMo RL benchmarks. The benchmarks covered NVIDIA DGX H100 and DGX GB200 systems. In a separate refit-only microbenchmark, Reshard refit took 4.3 seconds for the Qwen3.8 2.4T BF16 model.
Existing gather-and-broadcast refit implementations commonly reconstruct complete parameters
During reinforcement learning post-training, training trajectories are generated on the fly by running inference with the current policy model. NVIDIA describes a disaggregated setup in which inference and training run in parallel on separate GPU resources. The training GPUs update the policy, while the inference GPUs periodically receive the latest weights; NVIDIA calls that transfer refit.
Training and inference can distribute weights differently because they have different computational requirements and parallel mappings. Existing refit implementations commonly gather local shards on the training GPUs to reconstruct a complete parameter, broadcast that tensor to inference GPUs, then keep the shard each inference GPU needs. NVIDIA says the full-tensor intermediate creates redundant communication in both the gathering and broadcasting stages.
NVIDIA estimates that Kimi K3's 2.8 trillion parameters correspond to approximately 5.6 TB of BF16 weights, while Qwen 3.8 2.4T A95B's 2.4 trillion parameters correspond to approximately 4.8 TB. Counting the all-gather and broadcast stages, the estimated logical payload is approximately 11.2 TB for Kimi K3 and 9.6 TB for Qwen 3.8 2.4T A95B. The company says the payload for today's largest open weight models reaches around 10 TB per refit.
Reshard maps source shards to destination shards
NVIDIA frames refit as a transfer between two PyTorch distributed tensor (DTensor) layouts. Each layout describes the tensor's global shape, the logical arrangement of GPUs, and how the tensor is sharded or replicated across them. A Reshard operation takes source and destination DTensor descriptions and transfers the logical tensor from its source distribution into its destination distribution.
In NVIDIA's example, an 8-by-8 weight matrix is split by rows across two training GPUs and by columns across four inference GPUs. Each training GPU holds a 4-by-8 shard, and each inference GPU needs an 8-by-2 shard. Each destination can obtain its required shard by receiving a 4-by-2 region from each source, assembling its required shard without reconstructing the complete matrix.
NVIDIA describes NCCL M2N Reshard as an open-source reference implementation of the Reshard collective, available through the NVIDIA/nccl-extensions GitHub repo. It transfers a tensor distributed across M source GPUs into the layout required by N destination GPUs. In RL refit, the training GPUs form the source group and the inference GPUs form the destination group.
NCCL M2N Reshard eliminates redundant network transfers with NVLink exchanges
NCCL M2N Reshard calculates the exact intersections between source and destination shards, then maps those transfers onto the physical GPU network topology. It groups destination replicas within NVIDIA NVLink domains and distributes incoming contributions among replica ranks. The receiving GPUs exchange their subsets over NVLink to assemble complete shards.
For a TP4-to-TP2 x DP2 example, NVIDIA says direct point-to-point transfers send each source shard across the network to both data-parallel replicas, requiring eight network transfers. Reshard sends each of the four source shards over the network once, then uses NVLink exchanges to complete the replicas. NVIDIA says network transfers and local exchanges are pipelined to hide each other's latency.
NeMo RL applies Reshard to feed-forward projection weights
NVIDIA's reference implementation is provided in the NeMo RL framework. NVIDIA says NeMo RL is an open source framework designed to streamline and scale RL post-training. The framework needs to provide DTensor descriptions for each parameter in the training and inference instances. NVIDIA says the variety of layer types and sharding arrangements makes it challenging to cover every edge case through Reshard alone.
NVIDIA's NeMo RL design uses a hybrid path: feed-forward projection weights use Reshard, while all other small weights use conventional gather-and-broadcast. NVIDIA says feed-forward projection weights account for around 95% of parameters in the latest LLMs and dominate the refit payload. The company says the hybrid design retains the same functional coverage as conventional gather-and-broadcast refit.
NeMo RL benchmarks cover a range of models on NVIDIA DGX H100 and NVIDIA DGX GB200 systems
On NVIDIA DGX H100, Qwen 3 235B refit latency was 6.4 seconds with gather-and-broadcast and 1.9 seconds with Reshard. For DeepSeek v3 671B on the same platform, latency was 14.9 seconds with gather-and-broadcast and 2.6 seconds with Reshard. NVIDIA tested both models with different training-to-inference parallel layouts.
On NVIDIA DGX GB200, DeepSeek v3 671B refit latency was 6.1 seconds with gather-and-broadcast and 2.3 seconds with Reshard. Nemotron Ultra v3 on DGX GB200 had latency of 8.2 seconds with gather-and-broadcast and 2.3 seconds with Reshard. NVIDIA's NeMo RL refit guide and NCCL Reshard configuration requirements describe supported backends, layouts, and precision modes.