# Estimating Deep Learning energy consumption based on model architecture and training environment

arXiv: https://arxiv.org/abs/2307.05520 · HTML: https://mlsanity.com/paper/2307.05520/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 57 references that are in the map.
- First posted on arXiv 7 Jul 2023.

## Builds on (most cited first, 17 of 17)

- [Green AI](https://mlsanity.com/paper/1907.10597/) (2019)
- [Quantifying the Carbon Emissions of Machine Learning](https://mlsanity.com/paper/1910.09700/) (2019)
- [Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model](https://mlsanity.com/paper/2211.02001/) (2022)
- [Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learning](https://mlsanity.com/paper/2002.05651/) (2020)
- [The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink](https://mlsanity.com/paper/2204.05149/) (2022)
- [Compute Trends Across Three Eras of Machine Learning](https://mlsanity.com/paper/2202.05924/) (2022)
- [A Systematic Review of Green AI](https://mlsanity.com/paper/2301.11047/) (2023)
- [Counting Carbon: A Survey of Factors Influencing the Emissions of Machine Learning](https://mlsanity.com/paper/2302.08476/) (2023)
- [MixFaceNets: Extremely Efficient Face Recognition Networks](https://mlsanity.com/paper/2107.13046/) (2021)
- [How to estimate carbon footprint when training deep learning models? A guide and review](https://mlsanity.com/paper/2306.08323/) (2023)
- [Exploring the Carbon Footprint of Hugging Face's ML Models: A Repository Mining Study](https://mlsanity.com/paper/2305.11164/) (2023)
- [A survey on Variational Autoencoders from a GreenAI perspective](https://mlsanity.com/paper/2103.01071/) (2021)
- [Towards Accurate and Reliable Energy Measurement of NLP Models](https://mlsanity.com/paper/2010.05248/) (2020)
- [Uncovering Energy-Efficient Practices in Deep Learning Training: Preliminary Steps Towards Green AI](https://mlsanity.com/paper/2303.13972/) (2023)
- [The Ramifications of Making Deep Neural Networks Compact](https://mlsanity.com/paper/2006.15098/) (2020)
- [Precise Energy Consumption Measurements of Heterogeneous Artificial Intelligence Workloads](https://mlsanity.com/paper/2212.01698/) (2022)
- [Determining Chess Game State From an Image](https://mlsanity.com/paper/2104.14963/) (2021)

## Built on by (most cited first, 4 of 4)

- [A Synthesis of Green Architectural Tactics for ML-Enabled Systems](https://mlsanity.com/paper/2312.09610/) (2023)
- [Innovating for Tomorrow: The Convergence of SE and Green AI](https://mlsanity.com/paper/2406.18142/) (2024)
- [The More the Merrier? Navigating Accuracy vs. Energy Efficiency Design Trade-Offs in Ensemble Learning Systems](https://mlsanity.com/paper/2407.02914/) (2024)
- [AI of the People, by the People, for the People: A Social Choice Approach to Collective Control of Artificial Intelligence](https://mlsanity.com/paper/2605.16291/) (2026)

## Abstract (arXiv)

To raise awareness of the environmental impact of deep learning (DL), many studies estimate the energy use of DL systems. However, energy estimates during DL training often rely on unverified assumptions. This work addresses that gap by investigating how model architecture and training environment affect energy consumption. We train a variety of computer vision models and collect energy consumption and accuracy metrics to analyze their trade-offs across configurations. Our results show that selecting the right model-training environment combination can reduce training energy consumption by up to 80.68% with less than 2% loss in F1 score. We find a significant interaction effect between model and training environment: energy efficiency improves when GPU computational power scales with model complexity. Moreover, we demonstrate that common estimation practices, such as using FLOPs or GPU TDP, fail to capture these dynamics and can lead to substantial errors. To address these shortcomings, we propose the Stable Training Epoch Projection (STEP) and the Pre-training Regression-based Estimation (PRE) methods. Across evaluations, our methods outperform existing tools by a factor of two or more in estimation accuracy.