An Ai2 engineering report says its new GPU scheduler delivered 98% of the GPU hours its research teams were owed during a 30-day period. Ai2 calculates owed hours from each team’s allocation, capped at its actual demand; cluster occupancy remained at 98%.
Ai2 says it manages thousands of Nvidia H100, B200 and B300 GPUs across clusters of 88 to 1,024 GPUs, serving about 150 researchers. At any moment, submitted work requests two to three times the available capacity. That makes the queue policy consequential: a team that cannot start a small debugging run promptly may hold idle capacity in reserve or give every task the highest priority.
The institute says its old priority-based scheduler produced both behaviors. Some users kept jobs that did no work running on protected GPUs so they could connect when needed; eventually every scheduled job was marked “high” priority. Because jobs could opt out of preemption, engineers also had to negotiate the shutdown of long-running workloads on machines awaiting maintenance.
Ai2 replaced fixed priority decisions with GPU-time budgets set across a hierarchy of programs and research teams. Managers decide how much time each group can claim based on research priorities. A fair-share scheduler compares each group’s recent use with its budget over a rolling window, defaulting to seven days, and prioritizes work from groups that have used less than their share. Hierarchical fair-share scheduling already exists in systems such as Slurm; Ai2 says its change was to tie the hierarchy and budgets to its research organization.
The budget does not reserve a fixed rack for each team. If a funded project has no work ready, other jobs can use those otherwise unclaimed GPU hours, but the scheduler can interrupt them when a budgeted request arrives. Each submitted job also declares a minimum runtime: the period it needs to make meaningful progress before it can be interrupted. Ai2 caps this protected window at eight hours. Once it ends, a resumable job may continue if its group remains prioritized, or be interrupted and requeued, letting the scheduler rebalance the cluster and drain unhealthy machines without waiting for someone to stop a workload by hand.
In the reported 30-day period, Ai2 calculated each team’s owed time as its allocation capped at the hours it actually demanded. Teams received 98% of that total; 13 of 15 team allocations received at least 95%, and the lowest received 90%. Cluster occupancy, which Ai2 defines as the fraction of available time assigned to workloads, remained at 98% before and after the change. That measure is not the same as how much useful computation a GPU performs.
Queue waits also fell in the institute’s measurements. On its largest H100 cluster, median wait time went from five minutes to 24 seconds, while the 90th-percentile wait fell from 2.8 hours to 1.8 hours. For small debugging jobs, the 90th-percentile wait declined from two hours to 30 seconds; Ai2 cautioned that the earlier sample was smaller and more variable. It also reported 74% fewer repairs requiring a person in the loop after the scheduler began draining unhealthy hosts automatically.
Not every use case improved. Ai2 says researchers who kept interactive sessions open for up to a week could now lose them after the eight-hour protected window if their group had exceeded its allocation, forcing them to rebuild session state by hand. The institute plans a CPU-only development cluster and restorable sessions. It is also investigating whether fewer chances to interrupt running jobs could delay very large workloads.
These are results from one organization’s rollout, not an independent benchmark or evidence that the same policy will improve every cluster. The report measures access to GPU time, queue latency and maintenance work; it does not show lower energy use or faster model training.
Sources
- Ai2, “Impactful scheduling for GPU clusters,” October 9, 2026.
- Slurm Workload Manager, Fair Tree Fairshare Algorithm.
- Illustrative photo by Johan Fredriksson (Esquilo), “PDC server room,” Wikimedia Commons, under CC BY-SA 3.0. The unmodified photo shows a university server room, not Ai2 hardware.
