# Energy Efficiency of Training Neural Network Architectures: An Empirical Study

arXiv: https://arxiv.org/abs/2302.00967 · HTML: https://mlsanity.com/paper/2302.00967/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 25 references that are in the map.
- First posted on arXiv 2 Feb 2023.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm](https://mlsanity.com/paper/1712.01815/) (2017)
- [Green AI](https://mlsanity.com/paper/1907.10597/) (2019)
- [Carbon Emissions and Large Neural Network Training](https://mlsanity.com/paper/2104.10350/) (2021)
- [Integration of Convolutional Neural Networks in Mobile Applications](https://mlsanity.com/paper/2103.07286/) (2021)
- [Which Design Decisions in AI-enabled Mobile Applications Contribute to Greener AI?](https://mlsanity.com/paper/2109.15284/) (2021)

## Built on by (most cited first, 15 of 15)

- [Exploring the Carbon Footprint of Hugging Face's ML Models: A Repository Mining Study](https://mlsanity.com/paper/2305.11164/) (2023)
- [A Synthesis of Green Architectural Tactics for ML-Enabled Systems](https://mlsanity.com/paper/2312.09610/) (2023)
- [Green AI: A Preliminary Empirical Study on Energy Consumption in DL Models Across Different Runtime Infrastructures](https://mlsanity.com/paper/2402.13640/) (2024)
- [Towards green AI-based software systems: an architecture-centric approach (GAISSA)](https://mlsanity.com/paper/2307.09964/) (2023)
- [How to Sustainably Monitor ML-Enabled Systems? Accuracy and Energy Efficiency Tradeoffs in Concept Drift Detection](https://mlsanity.com/paper/2404.19452/) (2024)
- [Impact of ML Optimization Tactics on Greener Pre-Trained ML Models](https://mlsanity.com/paper/2409.12878/) (2024)
- [The More the Merrier? Navigating Accuracy vs. Energy Efficiency Design Trade-Offs in Ensemble Learning Systems](https://mlsanity.com/paper/2407.02914/) (2024)
- [Sustainable Machine Learning Retraining: Optimizing Energy Efficiency Without Compromising Accuracy](https://mlsanity.com/paper/2506.13838/) (2025)
- [EBA-AI: Ethics-Guided Bias-Aware AI for Efficient Underwater Image Enhancement and Coral Reef Monitoring](https://mlsanity.com/paper/2507.15036/) (2025)
- [Energy Consumption in Parallel Neural Network Training](https://mlsanity.com/paper/2508.07706/) (2025)
- [Insights into resource utilization of code small language models serving with runtime engines and execution providers](https://mlsanity.com/paper/2412.15441/) (2024)
- [Who Wins the Race? (R Vs Python) - An Exploratory Study on Energy Consumption of Machine Learning Algorithms](https://mlsanity.com/paper/2508.17344/) (2025)
- [Green AI: A systematic review and meta-analysis of its definitions, lifecycle models, hardware and measurement attempts](https://mlsanity.com/paper/2511.07090/) (2025)
- [LIFE -- an energy efficient advanced continual learning agentic AI framework for frontier systems](https://mlsanity.com/paper/2604.12874/) (2026)
- [TextTeacher: What Can Language Teach About Images?](https://mlsanity.com/paper/2605.22098/) (2026)

## Abstract (arXiv)

The evaluation of Deep Learning models has traditionally focused on criteria such as accuracy, F1 score, and related measures. The increasing availability of high computational power environments allows the creation of deeper and more complex models. However, the computations needed to train such models entail a large carbon footprint. In this work, we study the relations between DL model architectures and their environmental impact in terms of energy consumed and CO2 emissions produced during training by means of an empirical study using Deep Convolutional Neural Networks. Concretely, we study: (i) the impact of the architecture and the location where the computations are hosted on the energy consumption and emissions produced; (ii) the trade-off between accuracy and energy efficiency; and (iii) the difference on the method of measurement of the energy consumed using software-based and hardware-based tools.