The Pulse
Meta Puts Idle Inference Capacity to Work on Ad Ranking
Meta uses temporarily idle AI inference capacity to evaluate more ad candidates and route selected requests to more powerful models, the company says. The company describes safeguards for sudden capacity losses but publishes no numerical ad-performance lift.

Meta’s fleet can be constrained and idle at once
Meta’s AI inference fleet can be short of capacity overall even as pockets of computing power sit unused in particular regions, at certain times or on hardware that cannot serve the workload at hand, the company says. On October 12, the company described a system that pools some of that temporary slack for ad recommendations, where it can be reclaimed with little notice.
Source engineering.fb.com
“Inference fleets face a capacity paradox: They can be globally constrained while meaningful pockets of capacity remain locally idle.” Jinfu Leng, Rohith Menon, Nicolas Stier et al., Meta
That tension shapes the system’s design: extra ranking work can improve which ads the system considers, but it cannot be allowed to crowd out the capacity serving ordinary traffic. Meta’s approach puts otherwise idle resources to work without ordering additional capacity, the company says, though its post does not give a measured performance lift or a figure for how much capacity the system uses.
Two methods use different kinds of spare capacity
Meta calls its first method Rank More Ads, or RMA. Its ad system narrows millions of candidates through a series of ranking stages, with increasingly powerful machine-learning models assessing smaller groups. When spare capacity is available, RMA sends more candidates through those stages, increasing the chance of finding ads relevant to a person.
That extra work needs both CPU capacity for upstream tasks such as retrieval and feature processing, and accelerator capacity for model inference. Meta’s second method, Serve Better Models, or SBM, keeps the number of candidates steady but routes a larger share of requests to a more powerful model. Because it does not substantially increase upstream CPU work, SBM can use spare accelerator capacity that RMA cannot.
A request-value signal helps SBM decide which requests to route to the stronger model. An explore-and-exploit process measures the additional outcome for different request segments, then adjusts the allocation; it also preserves some exploratory traffic so the system can keep testing whether its choices are still effective.
Baseline ad serving gets a separate safety margin
The two methods protect ordinary traffic differently. SBM uses separate routing targets: dedicated replicas serve the baseline model, while replicas on opportunistic capacity handle requests selected for the stronger model. If that spare capacity contracts, the system lowers the share sent to the stronger model and sends those requests through the baseline path.
RMA uses a logical limit instead. Meta calculates how much extra candidate-ranking work the replicas on opportunistic capacity can support, then sets a global budget for that work. When capacity drops, RMA lowers its candidate-volume target; in both systems, the company says optional work is removed rather than shifted onto capacity reserved for baseline serving.
Central feedback loops adjust the targets as conditions change. RMA’s controller uses observed tail latency to raise or lower the number of candidates sent downstream, while SBM’s controller sets coverage for the stronger model and client-side routers apply it. Those central components operate on the order of minutes, leaving a gap when capacity disappears suddenly.
Local shedding handles sudden capacity losses
Meta’s faster fallback operates at both servers and clients. Servers can immediately remove lower-value opportunistic work if the available pool shrinks before clients receive updated targets; clients also monitor server status and shed extra work when they detect overload. Both kinds of shedding react within a few milliseconds, the company says.
Meta is working to expand the range of candidate volumes and stronger models the system can use, and to make models portable across more regions and accelerator types. It is also developing precomputation, which moves some work ahead of a live request, and an optimization-based system to allocate capacity among the available options. The October 12 post describes the mechanisms and intended benefits, but offers no numerical result for ad relevance, advertiser outcomes or capacity savings.
Sources
- Original source · Blog post Engineering at Meta engineering.fb.com