# HyperPower: Power- and Memory-Constrained Hyper-Parameter Optimization for Neural Networks

arXiv: https://arxiv.org/abs/1712.02446 · HTML: https://mlsanity.com/paper/1712.02446/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Dec 2017.

## Built on by (most cited first, 11 of 11)

- [A Systematic Review of Green AI](https://mlsanity.com/paper/2301.11047/) (2023)
- [Towards Green Automated Machine Learning: Status Quo and Future Directions](https://mlsanity.com/paper/2111.05850/) (2021)
- [A survey on multi-objective hyperparameter optimization algorithms for Machine Learning](https://mlsanity.com/paper/2111.13755/) (2021)
- [Exploring the Carbon Footprint of Hugging Face's ML Models: A Repository Mining Study](https://mlsanity.com/paper/2305.11164/) (2023)
- [A Survey on Approximate Edge AI for Energy Efficient Autonomous Driving Services](https://mlsanity.com/paper/2304.14271/) (2023)
- [Dynamic GPU Energy Optimization for Machine Learning Training Workloads](https://mlsanity.com/paper/2201.01684/) (2022)
- [PruMUX: Augmenting Data Multiplexing with Model Compression](https://mlsanity.com/paper/2305.14706/) (2023)
- [HyperTuner: A Cross-Layer Multi-Objective Hyperparameter Auto-Tuning Framework for Data Analytic Services](https://mlsanity.com/paper/2304.10051/) (2023)
- [Automated Computational Energy Minimization of ML Algorithms using Constrained Bayesian Optimization](https://mlsanity.com/paper/2407.05788/) (2024)
- [THOR: A Generic Energy Estimation Approach for On-Device Training](https://mlsanity.com/paper/2501.16397/) (2025)
- [CARB: A Characterization-Guided Framework for CNN Inference Cost Prediction and Deployment Screening](https://mlsanity.com/paper/2608.10506/) (2026)

## Abstract (arXiv)

While selecting the hyper-parameters of Neural Networks (NNs) has been so far treated as an art, the emergence of more complex, deeper architectures poses increasingly more challenges to designers and Machine Learning (ML) practitioners, especially when power and memory constraints need to be considered. In this work, we propose HyperPower, a framework that enables efficient Bayesian optimization and random search in the context of power- and memory-constrained hyper-parameter optimization for NNs running on a given hardware platform. HyperPower is the first work (i) to show that power consumption can be used as a low-cost, a priori known constraint, and (ii) to propose predictive models for the power and memory of NNs executing on GPUs. Thanks to HyperPower, the number of function evaluations and the best test error achieved by a constraint-unaware method are reached up to 112.99x and 30.12x faster, respectively, while never considering invalid configurations. HyperPower significantly speeds up the hyper-parameter optimization, achieving up to 57.20x more function evaluations compared to constraint-unaware methods for a given time interval, effectively yielding significant accuracy improvements by up to 67.6%.