The Pulse
Meta Details Networking for Gigawatt-Scale AI Clusters
Meta has outlined networking designs for its Prometheus AI cluster, including 102.4-terabit liquid-cooled switch racks and a fabric spanning multiple sites. It also says it is advancing open Ethernet standards and releasing its MetaRoCE sof
AI.info Team · Written with AI · fact-checked ·
Meta is pitching open networking for AI clusters while warning that a key part of switch design remains proprietary and fragmented. In a post published October 26, the company laid out networking work for its gigawatt-scale Prometheus cluster, including liquid-cooled 102.4-terabit switches and a new standard intended to reduce duplicated platform engineering.
The tension is practical: Meta says data-plane software can be portable, yet the lower-level switch platform layer still requires repeated work across hardware. Its response combines designs built with industry partners and proposals submitted to the Open Compute Project (OCP), rather than a single, uniform system.
102.4T switches move into liquid-cooled racks
Meta’s IcePack and IceCube design pairs a liquid-cooled network rack with switch blades rated at 102.4 terabits per second. The listed configurations use 128 800-gigabit Ethernet ports and either Broadcom Tomahawk 6C or NVIDIA Spectrum-6 switching silicon. Meta says the systems run FBOSS, its network operating system.
The company’s summit post describes the switches as non-blocking and presents them as part of a broader network stack, not a stand-alone speed upgrade. Meta also highlighted integrated optics work, including validation of 51.2T co-packaged optics and engineering for a 102.4T generation. The post describes that work as a look at the power, density, disaggregation and reliability trade-offs involved.
Eight planes, then a network across sites
For scale-out traffic inside the cluster, Meta described a Flattened Planar Fabric in which each GPU connects to eight network planes. The design changes the Clos topology from three tiers to two, uses NIC-level packet spraying and combines host and switch control planes. Meta says it is developing the fabric with NVIDIA and sharing it with cloud partners.
For traffic between buildings, campuses and regions, Meta pointed to Backend Aggregation, or BAG, a petabit-scale super-spine intended to connect those sites into a unified training network. The company also described regional backbones using Segment Routing over IPv6 (SRv6), running on FBOSS and the Switch Abstraction Interface (SAI). Meta’s post lists operating reliability, distance management and capacity planning among the engineering challenges discussed in its BAG session.
OSPA targets repeated platform work
Meta says the switch platform layer can add months of duplicated bring-up work per platform, even as standards such as SAI have made parts of switch software portable. Its proposed Open Switch Platform Architecture (OSPA) aims to standardize that layer. Meta, Microsoft and Alibaba are co-leading the OCP initiative, and Meta says the version 1.0 specification has been submitted to the organization.
The OCP’s OSPA announcement describes the draft as a vendor-neutral reference architecture and says it is undergoing Foundation review. It lists contributions from hardware and networking companies including Accton, Arista, Celestica, Cisco, Nvidia and Wistron. OSPA addresses platform management; it does not define routing or forwarding behavior.
Meta opens its transport software
The summit post also says Meta is making its MetaRoCE software implementations available as open source. The initial release is a user-space implementation on GitHub, with a DPDK-based version to follow, according to Meta. MetaRoCE is the company’s RDMA transport for AI networking, while other presentations addressed Ethernet scale-up, including Tahan for Meta’s MTIA 300 system and progress toward the next ESUN specification.
Meta posted recordings and slide decks for the summit sessions alongside its technical summary. The set of disclosures links its cluster architecture to specific hardware, open software and standards work; OSPA’s draft, meanwhile, is still under OCP review.