The Pulse
Modal Makes Multi-Node GPU Clusters Generally Available
Modal opened its multi-node GPU clusters to all workspaces on October 1, 2026, with gang scheduling, per-second billing and RDMA networking. The service uses a Python decorator to request coordinated GPU nodes, but requires full-GPU nodes a
AI.info Team · Written with AI ·
One request, several coordinated GPU nodes
Multi-node AI jobs need several machines to arrive together, but that coordination can make workloads harder to scale up and down. Modal says its newly generally available Clusters address that tension: developers can request a group of GPU containers through one Python decorator, while Modal handles placement and scheduling. The company announced the release on October 1, 2026, after 1.5 years of testing.
The service is available to all Modal workspaces. In its example, a developer adds @modal.clustered(size=4, rdma=True) to a function configured for eight B300 GPUs per container, requesting four coordinated containers. Modal says clusters draw from the same capacity pool as its other workloads, can be acquired in seconds, and are billed by the second.
Gang scheduling treats the cluster as one job
Modal’s existing scheduler makes decisions about individual nodes. For a multi-node job, the company says, it built a separate “gang scheduler” that considers the whole fleet, assigns all the requested nodes together within a zone and network, and starts more capacity if the existing fleet cannot fit the job.
That design matters because a partial cluster is not enough to start work that depends on all its nodes. Modal says the scheduler releases the job once the full group is ready, rather than running containers as they become available. The company also says its automated GPU health checks now extend to RDMA networking.
RDMA targets the traffic between GPUs
The product’s networking option is intended for workloads that move large amounts of data between machines, including distributed training and multi-node inference. Modal says setting rdma=True automates setup for PyTorch workloads using NCCL. Its announcement gives a maximum of 6.4 terabits per second; its documentation specifies that figure for B300 clusters, while other supported GPU types have 3.2 terabits per second of RDMA networking.
Modal’s implementation also addresses a problem in its secure container environment. The company says it added RDMA support to gVisor, which intercepts the networking-related system call and passes it to the host kernel; Modal says it has upstreamed those changes. Its description says the data can move directly between GPU memory and network interfaces without staging it in host RAM.
Runway cites elasticity; GPU limits still apply
Modal presents the clusters as useful for both training and serving. Runway CTO Kamil Sindi said the infrastructure challenge can make elastic multi-node inference difficult: “Multi-node inference is a challenging infrastructure problem. It makes elasticity hard, which is table stakes for inference workloads. Modal’s platform, with out of the box features like clustering and configurable autoscaling, has helped us rapidly scale multiple production inference workloads to meet growing demand.”
There are limits to the simplified interface. Modal’s documentation says each clustered container must request all GPUs on its host, and CPU-only clusters are not supported. Cluster size is also bounded by a workspace’s GPU limits, according to the company; the documentation describes groups of up to 256 devices, with larger clusters requiring contact with Modal.
For developers, the change is less about removing distributed-computing code than removing some of the infrastructure steps around it. Applications still need to coordinate work across containers, but Modal now offers an on-demand scheduler, network setup and billing model for acquiring the whole GPU group together.