# Power- and Fragmentation-aware Online Scheduling for GPU Datacenters

arXiv: https://arxiv.org/abs/2412.17484 · HTML: https://mlsanity.com/paper/2412.17484/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 24 references that are in the map.
- First posted on arXiv 23 Dec 2024.

## Builds on (most cited first, 5 of 5)

- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [Measuring the Carbon Intensity of AI in Cloud Instances](https://mlsanity.com/paper/2206.05229/) (2022)
- [Characterization and Prediction of Deep Learning Workloads in Large-Scale GPU Datacenters](https://mlsanity.com/paper/2109.01313/) (2021)
- [ANDREAS: Artificial intelligence traiNing scheDuler foR accElerAted resource clusterS](https://mlsanity.com/paper/2105.05080/) (2021)

## Built on by (most cited first, 1 of 1)

- [Adaptive GPU Resource Allocation for Multi-Agent Collaborative Reasoning in Serverless Environments](https://mlsanity.com/paper/2512.22149/) (2025)

## Abstract (arXiv)

The rise of Artificial Intelligence and Large Language Models is driving increased GPU usage in data centers for complex training and inference tasks, impacting operational costs, energy demands, and the environmental footprint of large-scale computing infrastructures. This work addresses the online scheduling problem in GPU datacenters, which involves scheduling tasks without knowledge of their future arrivals. We focus on two objectives: minimizing GPU fragmentation and reducing power consumption. GPU fragmentation occurs when partial GPU allocations hinder the efficient use of remaining resources, especially as the datacenter nears full capacity. A recent scheduling policy, Fragmentation Gradient Descent (FGD), leverages a fragmentation metric to address this issue. Reducing power consumption is also crucial due to the significant power demands of GPUs. To this end, we propose PWR, a novel scheduling policy to minimize power usage by selecting power-efficient GPU and CPU combinations. This involves a simplified model for measuring power consumption integrated into a Kubernetes score plugin. Through an extensive experimental evaluation in a simulated cluster, we show how PWR, when combined with FGD, achieves a balanced trade-off between reducing power consumption and minimizing GPU fragmentation.