The Pulse

AMD details GPU-initiated DMA transfers for multi-GPU workloads

AMD says GPU-initiated transfers using dedicated copy engines matched or exceeded default RCCL bandwidth in large-message tests while using fewer compute units. The feature is available in RCCL and rocSHMEM from ROCm 10.1, with source builds required.

AI.info Team · · fact-checked · against 1 source ↓ · Published 13 Nov 2026

GPU kernels can now initiate transfers between devices using dedicated copy engines in AMD's RCCL and rocSHMEM libraries. AMD says tests on eight Instinct MI350X GPUs showed comparable or higher large-message bandwidth than default RCCL operations while using one compute unit instead of 64.

The feature is available in the libraries from ROCm 10.1, with users required to build them from source. In a November 13 technical post, AMD described the mechanism and benchmark results for communication within a GPU node.

Separating copying from computation

GPU-initiated system DMA, or SDMA, lets a running GPU kernel issue transfer commands while dedicated engines perform the copying. Compute units coordinate the operation rather than executing the data movement themselves.

The post contrasts that approach with GPU load/store communication, in which compute units perform the copying, and host-initiated SDMA, in which the CPU starts the transfer. AMD says moving the copy work to dedicated engines leaves more compute resources available for kernels that combine computation and communication.

At a lower level, commands are placed in a ring buffer and a doorbell mechanism triggers their consumption by an SDMA engine. RCCL and rocSHMEM prepare the packets internally for device-initiated operations, rather than requiring device kernels to manage those details directly.

Results across two libraries

For the RCCL comparison, the researchers measured bus bandwidth against message size on a single node with eight MI350X GPUs. They compared the default load/store implementation with host-initiated and GPU-initiated SDMA for AlltoAll and Allgather operations.

The company reports that both SDMA implementations delivered comparable or slightly higher bandwidth than the default at the large message sizes tested. The SDMA configurations used symmetric memory registration, while the default implementation did not.

For AlltoAll, the GPU-initiated SDMA backend was used for messages greater than 32 MiB; smaller messages used vector load/store operations. For Allgather, the backend was used for messages of 512 KiB or larger.

AMD also compared rocSHMEM AlltoAll bandwidth across eight MI350X GPUs with SDMA enabled and disabled. The enabled test used one compute unit and one wavefront; the disabled test used one compute unit and 16 wavefronts. The enabled configuration delivered substantially higher bandwidth, according to the post.

Message size and setup

In a separate rocSHMEM putnbi benchmark between two MI350X GPUs, the company says a single compute unit achieved similar bandwidth with SDMA on or off at 128 KiB, although more compute units with SDMA off could achieve higher bandwidth at that size. For messages of 512 KiB or larger, SDMA enabled delivered the highest bandwidth among the tested configurations.

The current rocSHMEM implementation does not use SDMA for messages smaller than 128 KiB. At 64 KiB, the configuration with four wavefronts outperformed the single-wavefront configuration on the load/store path, the post says.

To use the feature, users must download RCCL or rocSHMEM from GitHub and build from source. RCCL requires symmetric memory registration and cuMem for its GPU-initiated networking support, and its SDMA backend is enabled through an environment variable.

AMD engineers repeated the experiments between five and seven times and reported one set of results because they were stable across runs. The RCCL results used a development version of the library with ROCm 7.0.2.2.0-16-9428210; the rocSHMEM tests used ROCm 10.1.0.