# Dynamic GPU Energy Optimization for Machine Learning Training Workloads

arXiv: https://arxiv.org/abs/2201.01684 · HTML: https://mlsanity.com/paper/2201.01684/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 47 references that are in the map.
- First posted on arXiv 5 Jan 2022.

## Builds on (most cited first, 8 of 8)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Benchmarking Graph Neural Networks](https://mlsanity.com/paper/2003.00982/) (2020)
- [HyperPower: Power- and Memory-Constrained Hyper-Parameter Optimization for Neural Networks](https://mlsanity.com/paper/1712.02446/) (2017)
- [Verified Instruction-Level Energy Consumption Measurement for NVIDIA GPUs](https://mlsanity.com/paper/2002.07795/) (2020)
- [Optimizing Deep Learning Inference on Embedded Systems Through Adaptive Model Selection](https://mlsanity.com/paper/1911.04946/) (2019)
- [Optimizing Sparse Matrix Multiplications for Graph Neural Networks](https://mlsanity.com/paper/2111.00352/) (2021)
- [Optimizing Streaming Parallelism on Heterogeneous Many-Core Architectures: A Machine Learning Based Approach](https://mlsanity.com/paper/2003.04294/) (2020)
- [Smart, Adaptive Energy Optimization for Mobile Web Interactions](https://mlsanity.com/paper/2005.00749/) (2020)

## Built on by (most cited first, 12 of 12)

- [DynamoLLM: Designing LLM Inference Clusters for Performance and Energy Efficiency](https://mlsanity.com/paper/2408.00741/) (2024)
- [Zeus: Understanding and Optimizing GPU Energy Consumption of DNN Training](https://mlsanity.com/paper/2208.06102/) (2022)
- [Reducing Energy Bloat in Large Model Training](https://mlsanity.com/paper/2312.06902/) (2023)
- [FusionAI: Decentralized Training and Deploying LLMs with Massive Consumer-Level GPUs](https://mlsanity.com/paper/2309.01172/) (2023)
- [Communication-Efficient Large-Scale Distributed Deep Learning: A Comprehensive Survey](https://mlsanity.com/paper/2404.06114/) (2024)
- [Deep Learning Workload Scheduling in GPU Datacenters: Taxonomy, Challenges and Vision](https://mlsanity.com/paper/2205.11913/) (2022)
- [PolyThrottle: Energy-efficient Neural Network Inference on Edge Devices](https://mlsanity.com/paper/2310.19991/) (2023)
- [Resource Allocation and Workload Scheduling for Large-Scale Distributed Deep Learning: A Survey](https://mlsanity.com/paper/2406.08115/) (2024)
- [Tensor-Aware Energy Accounting](https://mlsanity.com/paper/2311.11424/) (2023)
- [Online GPU Energy Optimization with Switching-Aware Bandits](https://mlsanity.com/paper/2410.11855/) (2024)
- [One Simple Trick for Improving the Performance of Energy-Limited Local Inference and Training](https://mlsanity.com/paper/2609.11936/) (2026)
- [A principled approach for energy-efficient training via phase-aware GPU frequency tuning](https://mlsanity.com/paper/2609.24205/) (2026)

## Abstract (arXiv)

GPUs are widely used to accelerate the training of machine learning workloads. As modern machine learning models become increasingly larger, they require a longer time to train, leading to higher GPU energy consumption. This paper presents GPOEO, an online GPU energy optimization framework for machine learning training workloads. GPOEO dynamically determines the optimal energy configuration by employing novel techniques for online measurement, multi-objective prediction modeling, and search optimization. To characterize the target workload behavior, GPOEO utilizes GPU performance counters. To reduce the performance counter profiling overhead, it uses an analytical model to detect the training iteration change and only collects performance counter data when an iteration shift is detected. GPOEO employs multi-objective models based on gradient boosting and a local search algorithm to find a trade-off between execution time and energy consumption. We evaluate the GPOEO by applying it to 71 machine learning workloads from two AI benchmark suites running on an NVIDIA RTX3080Ti GPU. Compared with the NVIDIA default scheduling strategy, GPOEO delivers a mean energy saving of 16.2% with a modest average execution time increase of 5.1%.