mlsanityData to 14 Oct 2026

Reducing Energy Bloat in Large Model Training

arXiv 2312.06902 ↗First posted 12 Dec 2023cs.LG cs.DCcited by 22

Not on the map. Why?

Abstract (arXiv)

Training large AI models on numerous GPUs consumes a massive amount of energy, making power delivery one of the largest limiting factors in building and operating datacenters for AI workloads. However, we observe that not all energy consumed during training directly contributes to end-to-end throughput; a significant portion can be removed without slowing down training. We call this portion energy bloat. In this work, we identify two independent sources of energy bloat in large model training and propose Perseus, a training system that mitigates both. To do this, Perseus obtains the time--energy tradeoff frontier of a large model training job using an efficient graph cut-based algorithm, and schedules computation energy consumption across time to reduce both types of energy bloat. Evaluation on large models, including GPT-3 and Bloom, shows that Perseus reduces the energy consumption of large model training by up to 30% without any throughput loss or hardware modification.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Builds on 37 of its 91 references that are in the map. First posted on arXiv 12 Dec 2023.

Builds on

37 of 91 references.

References · 37

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  6. 2020Scaling Laws for Neural Language Modelscited by 6,586
  7. 2016Wide Residual Networkscited by 3,070
  8. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  9. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  10. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
  11. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,254
  12. 2016Training Deep Nets with Sublinear Memory Costcited by 848
25 more
  1. 2021Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LMcited by 632
  2. 2019Green AIcited by 627
  3. 2021Carbon Emissions and Large Neural Network Trainingcited by 567
  4. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 559
  5. 2023The Falcon Series of Open Language Modelscited by 535
  6. 2019Quantifying the Carbon Emissions of Machine Learningcited by 521
  7. 2017Efficient Processing of Deep Neural Networks: A Tutorial and Surveycited by 434
  8. 2022Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Modelcited by 265
  9. 2020Memory-Efficient Pipeline-Parallel DNN Trainingcited by 134
  10. 2020Serving DNNs like Clockwork: Performance Predictability from the Bottom Upcited by 120
  11. 2022Measuring the Carbon Intensity of AI in Cloud Instancescited by 111
  12. 2020DAPPLE: A Pipelined Data Parallel Approach for Training Large Modelscited by 101
  13. 2019Analysis of Large-Scale Multi-Tenant GPU Clusters for DNN Training Workloadscited by 79
  14. 2022Zeus: Understanding and Optimizing GPU Energy Consumption of DNN Trainingcited by 79
  15. 2019PUMA: A Programmable Ultra-efficient Memristor-based Accelerator for Machine Learning Inferencecited by 61
  16. 2022Bamboo: Making Preemptible Instances Resilient for Affordable Training of Large DNNscited by 49
  17. 2023Oobleck: Resilient Distributed Training of Large Models Using Pipeline Templatescited by 40
  18. 2020Analyzing and Mitigating Data Stalls in DNN Trainingcited by 36
  19. 2021Understanding Data Storage and Ingestion for Large-Scale Deep Recommendation Model Trainingcited by 28
  20. 2018On the Computational Inefficiency of Large Batch Sizes for Stochastic Gradient Descentcited by 25
  21. 2022PipeFisher: Efficient Training of Large Language Models Using Pipelining and Fisher Information Matricescited by 24
  22. 2022Dynamic GPU Energy Optimization for Machine Learning Training Workloadscited by 12
  23. 2023POLCA: Power Oversubscription in LLM Cloud Providerscited by 9
  24. 2020A Data-Driven Frequency Scaling Approach for Deadline-aware Energy Efficient Scheduling on Graphics Processing Units (GPUs)cited by 2
  25. 2023Chasing Low-Carbon Electricity for Practical and Sustainable DNN Trainingcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 7; 2026: 14.

22 papers.

Sort:

By area:

  1. 2025The ML.ENERGY Benchmark: Toward Automated Inference Energy Measurement and Optimizationcited by 33
  2. 2025Characterizing LLM Inference Energy-Performance Tradeoffs across Workloads and GPU Scalingcited by 19
  3. 2026Where Do the Joules Go? Diagnosing Inference Energy Consumptioncited by 10
  4. 2026DualScale: Energy-Efficient Disaggregated LLM Serving via Phase-Aware Placement and DVFScited by 5
  5. 2026Kareus: Joint Reduction of Dynamic and Static Energy in Large Model Trainingcited by 3
  6. 2025ELANA: A Simple Energy and Latency Analyzer for LLMscited by 2
  7. 2024Input-Dependent Power Usage in GPUscited by 1
  8. 2025Good things come in small packages: Should we build AI clusters with Lite-GPUs?cited by 1
  9. 2025One Search Fits All: Pareto-Optimal Eco-Friendly Model Selectioncited by 1
  10. 2025Active Inference AI Systems for Scientific Discoverycited by 1
  11. 2026Nalar: An agent serving frameworkcited by 1
  12. 2026ViBE: Co-Optimizing Workload Skew and Hardware Variability for MoE Servingcited by 1
10 more
  1. 2025Magneton: Optimizing Energy Efficiency of ML Systems via Differential Energy Debuggingcited by 0
  2. 2026Optimising for Energy Efficiency and Performance in Machine Learningcited by 0
  3. 2026Designing Datacenter Power Delivery Hierarchies for the AI Eracited by 0
  4. 2026GEM: GPU-Variability-Aware Expert to GPU Mapping for MoE Systemscited by 0
  5. 2026Evaluation of ML Resource Utilization Requires Model Life Cycle Assessmentcited by 0
  6. 2026Efficient Resource Optimization for Split Federated Learningcited by 0
  7. 2026Phase-Decoupled, Model-Calibrated Power Control for Disaggregated LLM Servingcited by 0
  8. 2026Characterizing Job Power Elasticity for Power-Flexible AI Trainingcited by 0
  9. 2026Minibatch persistency, eight years later: what batch reuse costs in steps and joules, and what it saves in datacited by 0
  10. 2026A principled approach for energy-efficient training via phase-aware GPU frequency tuningcited by 0