mlsanityData to 7 Oct 2026

Zeus: Understanding and Optimizing GPU Energy Consumption of DNN Training

arXiv 2208.06102 ↗First posted 12 Aug 2022cs.LG cs.AI cs.DCcited by 78

Not on the map. Why?

Abstract (arXiv)

Training deep neural networks (DNNs) is becoming increasingly more resource- and energy-intensive every year. Unfortunately, existing works primarily focus on optimizing DNN training for faster completion, often without considering the impact on energy efficiency. In this paper, we observe that common practices to improve training performance can often lead to inefficient energy usage. More importantly, we demonstrate that there is a tradeoff between energy consumption and performance optimization. To this end, we propose Zeus, an optimization framework to navigate this tradeoff by automatically finding optimal job- and GPU-level configurations for recurring DNN training jobs. Zeus uses an online exploration-exploitation approach in conjunction with just-in-time energy profiling, averting the need for expensive offline measurements, while adapting to data drifts over time. Our evaluation shows that Zeus can improve the energy efficiency of DNN training by 15.3%-75.8% for diverse workloads.

Key facts. Cited by 78 papers in the map (data through 7 Oct 2026). Builds on 47 of its 103 references that are in the map. First posted on arXiv 12 Aug 2022.

Builds on

47 of 103 references.

References · 47

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  7. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  8. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  9. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  10. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
  11. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  12. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
35 more
  1. 2017Neural Collaborative Filteringcited by 1,108
  2. 2016An overview of gradient descent optimization algorithmscited by 1,021
  3. 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 688
  4. 2021Carbon Emissions and Large Neural Network Trainingcited by 561
  5. 2019Quantifying the Carbon Emissions of Machine Learningcited by 516
  6. 2020Learning under Concept Drift: A Reviewcited by 470
  7. 2017A Tutorial on Thompson Samplingcited by 420
  8. 2014Deep Speech: Scaling up end-to-end speech recognitioncited by 354
  9. 2017Don't Decay the Learning Rate, Increase the Batch Sizecited by 313
  10. 2017Train longer, generalize better: closing the generalization gap in large batch training of neural networkscited by 278
  11. 2021GRAD-MATCH: Gradient Matching based Data Subset Selection for Efficient Deep Model Trainingcited by 269
  12. 2018An Empirical Model of Large-Batch Trainingcited by 268
  13. 2020Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learningcited by 246
  14. 2021Sustainable AI: Environmental Implications, Challenges and Opportunitiescited by 242
  15. 2018A System for Massively Parallel Hyperparameter Tuningcited by 201
  16. 2020Ansor: Generating High-Performance Tensor Programs for Deep Learningcited by 199
  17. 2020Carbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Modelscited by 168
  18. 2022Measuring the Carbon Intensity of AI in Cloud Instancescited by 110
  19. 2021GSPMD: General and Scalable Parallelization for ML Computation Graphscited by 109
  20. 2019The Architectural Implications of Facebook's DNN-based Personalized Recommendationcited by 83
  21. 2022MetaShift: A Dataset of Datasets for Evaluating Contextual Distribution Shifts and Training Conflictscited by 81
  22. 2020EdgeBERT: Sentence-Level Energy Optimizations for Latency-Aware Multi-Task NLP Inferencecited by 69
  23. 2020Pollux: Co-adaptive Cluster Scheduling for Goodput-Optimized Deep Learningcited by 68
  24. 2020Bao: Learning to Steer Query Optimizerscited by 62
  25. 2020Learning Rates as a Function of Batch Size: A Random Matrix Theory Approach to Neural Network Trainingcited by 50
  26. 2019Blink: Fast and Generic Collectives for Distributed MLcited by 34
  27. 2020Hardware and Software Optimizations for Accelerating Deep Neural Networks: Survey of Current Trends, Challenges, and the Road Aheadcited by 34
  28. 2018On the Computational Inefficiency of Large Batch Sizes for Stochastic Gradient Descentcited by 25
  29. 2018Analysis of DAWNBench, a Time-to-Accuracy Machine Learning Performance Benchmarkcited by 21
  30. 2020Towards Accurate and Reliable Energy Measurement of NLP Modelscited by 16
  31. 2022Dynamic GPU Energy Optimization for Machine Learning Training Workloadscited by 12
  32. 2022Treehouse: A Case For Carbon-Aware Datacenter Softwarecited by 12
  33. 2019ALERT: Accurate Learning for Energy and Timelinesscited by 9
  34. 2020Verified Instruction-Level Energy Consumption Measurement for NVIDIA GPUscited by 9
  35. 2016A Survey and Measurement Study of GPU DVFS on Energy Conservationcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 13; 2024: 9; 2025: 25; 2026: 30.

78 papers.

Sort:

By area:

  1. 2024DynamoLLM: Designing LLM Inference Clusters for Performance and Energy Efficiencycited by 106
  2. 2024Characterization of Large Language Model Development in the Datacentercited by 46
  3. 2022Compute-Efficient Deep Learning: Algorithmic Trends and Opportunitiescited by 38
  4. 2024MELTing point: Mobile Evaluation of Language Transformerscited by 35
  5. 2025The ML.ENERGY Benchmark: Toward Automated Inference Energy Measurement and Optimizationcited by 31
  6. 2025Measuring the environmental impact of delivering AI at Google Scalecited by 31
  7. 2023Reducing Energy Bloat in Large Model Trainingcited by 22
  8. 2024The Price of Prompting: Profiling Energy Use in Large Language Models Inferencecited by 22
  9. 2023FusionAI: Decentralized Training and Deploying LLMs with Massive Consumer-Level GPUscited by 20
  10. 2025Characterizing LLM Inference Energy-Performance Tradeoffs across Workloads and GPU Scalingcited by 19
  11. 2023Thermodynamic Computing System for AI Applicationscited by 16
  12. 2024MLPerf Power: Benchmarking the Energy Efficiency of Machine Learning Systems from Microwatts to Megawatts for Sustainable AIcited by 15
38 more
  1. 2023DVFO: Learning-Based DVFS for Energy-Efficient Edge-Cloud Collaborative Inferencecited by 13
  2. 2025Intelligence per Watt: Measuring Intelligence Efficiency of Local AIcited by 13
  3. 2025From Prompts to Power: Measuring the Energy Footprint of LLM Inferencecited by 12
  4. 2025MoE-CAP: Benchmarking Cost, Accuracy and Performance of Sparse Mixture-of-Experts Systemscited by 11
  5. 2023Cost-effective On-device Continual Learning over Memory Hierarchy with Mirocited by 10
  6. 2026Where Do the Joules Go? Diagnosing Inference Energy Consumptioncited by 10
  7. 2023POLCA: Power Oversubscription in LLM Cloud Providerscited by 8
  8. 2025An efficient probabilistic hardware architecture for diffusion-like modelscited by 8
  9. 2023PolyThrottle: Energy-efficient Neural Network Inference on Edge Devicescited by 5
  10. 2025Characterizing the Efficiency of Distributed Training: A Power, Performance, and Thermal Perspectivecited by 5
  11. 2026Watt Counts: Energy-Aware Benchmark for Sustainable LLM Inference on Heterogeneous GPU Architecturescited by 5
  12. 2023Real-Time Semantic Segmentation: A Brief Survey & Comparative Study in Remote Sensingcited by 4
  13. 2023DeepEn2023: Energy Datasets for Edge Artificial Intelligencecited by 4
  14. 2024AxiomVision: Accuracy-Guaranteed Adaptive Visual Model Selection for Perspective-Aware Video Analyticscited by 4
  15. 2024Generative AI on the Edge: Architecture and Performance Evaluationcited by 4
  16. 2026DualScale: Energy-Efficient Disaggregated LLM Serving via Phase-Aware Placement and DVFScited by 4
  17. 2025Green Prompting: Characterizing Prompt-driven Energy Costs of LLM Inferencecited by 3
  18. 2025Small is Sufficient: Reducing the World AI Energy Consumption Through Model Selectioncited by 3
  19. 2025From Tokens to Layers: Redefining Stall-Free Scheduling for MoE Serving with Layered Prefillcited by 3
  20. 2026Kareus: Joint Reduction of Dynamic and Static Energy in Large Model Trainingcited by 3
  21. 2023Chasing Low-Carbon Electricity for Practical and Sustainable DNN Trainingcited by 2
  22. 2025E4: Energy-Efficient DNN Inference for Edge Video Analytics Via Early-Exit and DVFScited by 2
  23. 2025Tu(r)ning AI Green: Exploring Energy Efficiency Cascading with Orthogonal Optimizationscited by 2
  24. 2025FAST: Topology-Aware Frequency-Domain Distribution Matching for Coreset Selectioncited by 2
  25. 2025ELANA: A Simple Energy and Latency Analyzer for LLMscited by 2
  26. 2026Characterizing Performance-Energy Trade-offs of Large Language Models in Multi-Request Workflowscited by 2
  27. 2026PowerSlider: Exploiting Phase Asymmetry for LLM Serving under Demand Responsecited by 2
  28. 2023Tensor-Aware Energy Accountingcited by 1
  29. 2023Venn: Resource Management for Collaborative Learning Jobscited by 1
  30. 2024Input-Dependent Power Usage in GPUscited by 1
  31. 2024Automating Energy-Efficient GPU Kernel Generation: A Fast Search-Based Compilation Approachcited by 1
  32. 2025SMDP-Based Dynamic Batching for Improving Responsiveness and Energy Efficiency of Batch Servicescited by 1
  33. 2025One Search Fits All: Pareto-Optimal Eco-Friendly Model Selectioncited by 1
  34. 2025Energy Consumption in Parallel Neural Network Trainingcited by 1
  35. 2026KAIROS: Stateful, Context-Aware Power-Efficient Agentic Inference Servingcited by 1
  36. 2026The Illusion of Power Capping in LLM Decode: A Phase-Aware Energy Characterisation Across Attention Architecturescited by 1
  37. 2026PALS: Power-Aware LLM Serving for Mixture-of-Experts Modelscited by 1
  38. 2026XWind: A Cross-site Router for Large Language Model Inference Serving at Renewable Energy Farmscited by 1

50 most cited of 78 All 78 (JSON)