# Uncovering Energy-Efficient Practices in Deep Learning Training: Preliminary Steps Towards Green AI

arXiv: https://arxiv.org/abs/2303.13972 · HTML: https://mlsanity.com/paper/2303.13972/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 45 references that are in the map.
- First posted on arXiv 24 Mar 2023.

## Builds on (most cited first, 12 of 12)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Green AI](https://mlsanity.com/paper/1907.10597/) (2019)
- [Sustainable AI: Environmental Implications, Challenges and Opportunities](https://mlsanity.com/paper/2111.00364/) (2021)
- [Predictive Entropy Search for Efficient Global Optimization of Black-box Functions](https://mlsanity.com/paper/1406.2541/) (2014)
- [Designing Energy-Efficient Convolutional Neural Networks using Energy-Aware Pruning](https://mlsanity.com/paper/1611.05128/) (2016)
- [Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets](https://mlsanity.com/paper/1605.07079/) (2016)
- [Grid Search, Random Search, Genetic Algorithm: A Big Comparison for NAS](https://mlsanity.com/paper/1912.06059/) (2019)
- [A Systematic Review of Green AI](https://mlsanity.com/paper/2301.11047/) (2023)
- [Data-Centric Green AI: An Exploratory Empirical Study](https://mlsanity.com/paper/2204.02766/) (2022)
- [An Equivalence of Fully Connected Layer and Convolutional Layer](https://mlsanity.com/paper/1712.01252/) (2017)
- [Catalog of Energy Patterns for Mobile Applications](https://mlsanity.com/paper/1901.03302/) (2019)

## Built on by (most cited first, 16 of 16)

- [Exploring the Carbon Footprint of Hugging Face's ML Models: A Repository Mining Study](https://mlsanity.com/paper/2305.11164/) (2023)
- [A Synthesis of Green Architectural Tactics for ML-Enabled Systems](https://mlsanity.com/paper/2312.09610/) (2023)
- [Watt For What: Rethinking Deep Learning's Energy-Performance Relationship](https://mlsanity.com/paper/2310.06522/) (2023)
- [Efficiency is Not Enough: A Critical Perspective of Environmentally Sustainable AI](https://mlsanity.com/paper/2309.02065/) (2023)
- [Green AI: A Preliminary Empirical Study on Energy Consumption in DL Models Across Different Runtime Infrastructures](https://mlsanity.com/paper/2402.13640/) (2024)
- [Estimating Deep Learning energy consumption based on model architecture and training environment](https://mlsanity.com/paper/2307.05520/) (2023)
- [Batching for Green AI -- An Exploratory Study on Inference](https://mlsanity.com/paper/2307.11434/) (2023)
- [Involution Fused ConvNet for Classifying Eye-Tracking Patterns of Children with Autism Spectrum Disorder](https://mlsanity.com/paper/2401.03575/) (2024)
- [Innovating for Tomorrow: The Convergence of SE and Green AI](https://mlsanity.com/paper/2406.18142/) (2024)
- [How Do Companies Manage the Environmental Sustainability of AI? An Interview Study About Green AI Efforts and Regulations](https://mlsanity.com/paper/2505.07317/) (2025)
- [Energy-Efficient Green AI Architectures for Circular Economies Through Multi-Layered Sustainable Resource Optimization Framework](https://mlsanity.com/paper/2506.12262/) (2025)
- [Energy Consumption in Parallel Neural Network Training](https://mlsanity.com/paper/2508.07706/) (2025)
- [Are Trees Really Green? A Detection Approach of IoT Malware Attacks](https://mlsanity.com/paper/2506.07836/) (2025)
- [Optimising for Energy Efficiency and Performance in Machine Learning](https://mlsanity.com/paper/2601.08991/) (2026)
- [The Energy Impact of Domain Model Design in Classical Planning](https://mlsanity.com/paper/2601.21967/) (2026)
- [The Energy Consumption of Transformer Fine-Tuning: A Roofline-Inspired Scaling Model](https://mlsanity.com/paper/2606.23546/) (2026)

## Abstract (arXiv)

Modern AI practices all strive towards the same goal: better results. In the context of deep learning, the term "results" often refers to the achieved accuracy on a competitive problem set. In this paper, we adopt an idea from the emerging field of Green AI to consider energy consumption as a metric of equal importance to accuracy and to reduce any irrelevant tasks or energy usage. We examine the training stage of the deep learning pipeline from a sustainability perspective, through the study of hyperparameter tuning strategies and the model complexity, two factors vastly impacting the overall pipeline's energy consumption. First, we investigate the effectiveness of grid search, random search and Bayesian optimisation during hyperparameter tuning, and we find that Bayesian optimisation significantly dominates the other strategies. Furthermore, we analyse the architecture of convolutional neural networks with the energy consumption of three prominent layer types: convolutional, linear and ReLU layers. The results show that convolutional layers are the most computationally expensive by a strong margin. Additionally, we observe diminishing returns in accuracy for more energy-hungry models. The overall energy consumption of training can be halved by reducing the network complexity. In conclusion, we highlight innovative and promising energy-efficient practices for training deep learning models. To expand the application of Green AI, we advocate for a shift in the design of deep learning models, by considering the trade-off between energy efficiency and accuracy.