# The Energy Cost of Execution-Idle in GPU Clusters

arXiv: https://arxiv.org/abs/2604.04745 · HTML: https://mlsanity.com/paper/2604.04745/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Apr 2026.

## Built on by (most cited first, 3 of 3)

- [Characterization of Request and Token Energy Costs for LLM Inference Workloads on GPU Platforms](https://mlsanity.com/paper/2608.28044/) (2026)
- [WattGPU: Predicting Inference Power and Latency on Unseen GPUs and LLMs](https://mlsanity.com/paper/2607.02391/) (2026)
- [A principled approach for energy-efficient training via phase-aware GPU frequency tuning](https://mlsanity.com/paper/2609.24205/) (2026)

## Abstract (arXiv)

GPUs are becoming a major contributor to data center power, yet unlike CPUs, they can remain at high power even when visible activity is near zero. We call this state execution-idle. Using per-second telemetry from a large academic AI cluster, we characterize execution-idle as a recurring low-activity yet high-power state in real deployments. Across diverse workloads and multiple GPU generations, it accounts for 19.7% of in-execution time and 10.7% of energy. This suggests a need to both reduce the cost of execution-idle and reduce exposure to it. We therefore build two prototypes: one uses automatic downscaling during execution-idle, and the other uses load imbalance to reduce exposure, both with performance trade-offs. These findings suggest that future energy-efficient GPU systems should treat execution-idle as a first-class operating state.