# Automated Computational Energy Minimization of ML Algorithms using Constrained Bayesian Optimization

arXiv: https://arxiv.org/abs/2407.05788 · HTML: https://mlsanity.com/paper/2407.05788/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 24 references that are in the map.
- First posted on arXiv 8 Jul 2024.

## Builds on (most cited first, 8 of 8)

- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice](https://mlsanity.com/paper/2007.15745/) (2020)
- [Designing Energy-Efficient Convolutional Neural Networks using Energy-Aware Pruning](https://mlsanity.com/paper/1611.05128/) (2016)
- [Bayesian Optimization with Unknown Constraints](https://mlsanity.com/paper/1403.5607/) (2014)
- [A General Framework for Constrained Bayesian Optimization using Information-based Search](https://mlsanity.com/paper/1511.09422/) (2015)
- [Fair Bayesian Optimization](https://mlsanity.com/paper/2006.05109/) (2020)
- [A multi-objective perspective on jointly tuning hardware and hyperparameters](https://mlsanity.com/paper/2106.05680/) (2021)
- [HyperPower: Power- and Memory-Constrained Hyper-Parameter Optimization for Neural Networks](https://mlsanity.com/paper/1712.02446/) (2017)

## Abstract (arXiv)

Bayesian optimization (BO) is an efficient framework for optimization of black-box objectives when function evaluations are costly and gradient information is not easily accessible. BO has been successfully applied to automate the task of hyperparameter optimization (HPO) in machine learning (ML) models with the primary objective of optimizing predictive performance on held-out data. In recent years, however, with ever-growing model sizes, the energy cost associated with model training has become an important factor for ML applications. Here we evaluate Constrained Bayesian Optimization (CBO) with the primary objective of minimizing energy consumption and subject to the constraint that the generalization performance is above some threshold. We evaluate our approach on regression and classification tasks and demonstrate that CBO achieves lower energy consumption without compromising the predictive performance of ML models.