The Pulse
Ai2 Replaces GPU Priorities With Research-Time Budgets
Ai2 says its new GPU scheduler cut median queue waits on its largest H100 cluster from five minutes to 24 seconds while keeping occupancy at 98%. The system allocates GPU-time budgets to research projects and can preempt eligible jobs when
AI.info Team · Written with AI ·
Twenty-four seconds: that is the median queue wait on Ai2’s largest H100 cluster after the institute changed how it assigns GPU time, down from five minutes. The new scheduler replaces a priority-based system with project budgets, hierarchical fair-share allocation and rules allowing eligible workloads to be interrupted and requeued. Ai2 published its account of the change on October 9, 2026, along with results from a 30-day test.
Why Ai2 dropped its priority system
Ai2 says it manages thousands of NVIDIA H100, B200 and B300 GPUs across clusters ranging from 88 to 1,024 GPUs, serving about 150 internal researchers. Submitted workloads call for two to three times the available GPU capacity, the organization says. “At any moment in time we have outstanding requests for 2-3x more GPUs than are available,” writes Kyle Wiggers, credited on the post as Ai2Comms.
Under the old system, users could opt to protect workloads from preemption, while priority labels shaped which jobs ran first. Ai2 says that over time every scheduled workload reached HIGH priority, leaving lower levels starved. Researchers also parked idle, no-op workloads on GPUs so they could connect quickly when needed—a behavior the post calls “squatting.”
Budgets set the claim; the scheduler handles the queue
Managers now allocate GPU-time budgets through a hierarchy of research programs, projects and researchers. The allocations give teams a defined share of capacity over time without reserving particular GPUs that might sit idle when a project has no ready jobs. Every protected request draws against the relevant budget, making idle jobs count against the group that submitted them.
The scheduler compares each group’s recent GPU occupancy with its allocation, using a seven-day lookback by default, and favors workloads from groups that have used less than their share. Submitted jobs must also state a minimum runtime—the time needed to make meaningful progress. They are protected from preemption for that period, then may continue running or be interrupted and requeued if they can resume.
Ai2 caps the protected minimum runtime at eight hours. Jobs that declare a minimum runtime of zero remain interruptible and do not draw on a team’s budget. The organization says the change also lets unhealthy hosts drain workloads after their protected runtimes, cutting repairs that require a person to intervene by 74%.
Queue waits fell, while occupancy held at 98%
Over the 30-day test, Ai2 says teams received 98% of the GPU hours they were owed, counting each allocation only up to the team’s actual demand. Thirteen of 15 team allocations received at least 95% of their owed time; the lowest received 90%. Cluster occupancy stayed at 98% before and after the change, and 18% of delivered GPU time went to unallocated work when budgeted projects were not ready to run.
On the largest H100 cluster, median queue wait fell from five minutes to 24 seconds, while the 90th-percentile wait dropped from 2.8 hours to 1.8 hours. Debug workloads—small jobs used to check whether code launches or fails early—saw their 90th-percentile wait fall from two hours to 30 seconds. Ai2 cautions that the baseline had fewer such jobs, adding more variance to that comparison.
Interactive sessions expose the trade-off
Not all researchers benefited. Interactive sessions for data analysis and code testing could previously remain on the old system for up to a week; under the new rules, they can become preemptible after the eight-hour protection cap if they exceed their allocation. Ai2 says users sometimes had to rebuild session state after interruption.
The organization says it is planning a CPU-only cluster near its on-premises storage for development and data-preparation sessions, along with restorable sessions that can move elsewhere after preemption. It is also investigating whether minimum-runtime protections could make it harder to assemble enough GPUs for the largest jobs, a possible capacity-fragmentation problem it says it is testing in simulation and production.