# Reducing Energy Bloat in Large Model Training

arXiv: https://arxiv.org/abs/2312.06902 · HTML: https://mlsanity.com/paper/2312.06902/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 91 references that are in the map.
- First posted on arXiv 12 Dec 2023.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism](https://mlsanity.com/paper/1909.08053/) (2019)
- [On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima](https://mlsanity.com/paper/1609.04836/) (2016)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Training Deep Nets with Sublinear Memory Cost](https://mlsanity.com/paper/1604.06174/) (2016)
- [Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM](https://mlsanity.com/paper/2104.04473/) (2021)
- [Green AI](https://mlsanity.com/paper/1907.10597/) (2019)
- [Carbon Emissions and Large Neural Network Training](https://mlsanity.com/paper/2104.10350/) (2021)
- [Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model](https://mlsanity.com/paper/2201.11990/) (2022)
- [The Falcon Series of Open Language Models](https://mlsanity.com/paper/2311.16867/) (2023)
- [Quantifying the Carbon Emissions of Machine Learning](https://mlsanity.com/paper/1910.09700/) (2019)
- [Efficient Processing of Deep Neural Networks: A Tutorial and Survey](https://mlsanity.com/paper/1703.09039/) (2017)
- [Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model](https://mlsanity.com/paper/2211.02001/) (2022)
- [Memory-Efficient Pipeline-Parallel DNN Training](https://mlsanity.com/paper/2006.09503/) (2020)
- [Serving DNNs like Clockwork: Performance Predictability from the Bottom Up](https://mlsanity.com/paper/2006.02464/) (2020)
- [Measuring the Carbon Intensity of AI in Cloud Instances](https://mlsanity.com/paper/2206.05229/) (2022)
- [DAPPLE: A Pipelined Data Parallel Approach for Training Large Models](https://mlsanity.com/paper/2007.01045/) (2020)
- [Analysis of Large-Scale Multi-Tenant GPU Clusters for DNN Training Workloads](https://mlsanity.com/paper/1901.05758/) (2019)
- [Zeus: Understanding and Optimizing GPU Energy Consumption of DNN Training](https://mlsanity.com/paper/2208.06102/) (2022)
- [PUMA: A Programmable Ultra-efficient Memristor-based Accelerator for Machine Learning Inference](https://mlsanity.com/paper/1901.10351/) (2019)
- [Bamboo: Making Preemptible Instances Resilient for Affordable Training of Large DNNs](https://mlsanity.com/paper/2204.12013/) (2022)
- [Oobleck: Resilient Distributed Training of Large Models Using Pipeline Templates](https://mlsanity.com/paper/2309.08125/) (2023)
- [Analyzing and Mitigating Data Stalls in DNN Training](https://mlsanity.com/paper/2007.06775/) (2020)

## Built on by (most cited first, 22 of 22)

- [The ML.ENERGY Benchmark: Toward Automated Inference Energy Measurement and Optimization](https://mlsanity.com/paper/2505.06371/) (2025)
- [Characterizing LLM Inference Energy-Performance Tradeoffs across Workloads and GPU Scaling](https://mlsanity.com/paper/2501.08219/) (2025)
- [Where Do the Joules Go? Diagnosing Inference Energy Consumption](https://mlsanity.com/paper/2601.22076/) (2026)
- [DualScale: Energy-Efficient Disaggregated LLM Serving via Phase-Aware Placement and DVFS](https://mlsanity.com/paper/2602.18755/) (2026)
- [Kareus: Joint Reduction of Dynamic and Static Energy in Large Model Training](https://mlsanity.com/paper/2601.17654/) (2026)
- [ELANA: A Simple Energy and Latency Analyzer for LLMs](https://mlsanity.com/paper/2512.09946/) (2025)
- [Input-Dependent Power Usage in GPUs](https://mlsanity.com/paper/2409.18324/) (2024)
- [Good things come in small packages: Should we build AI clusters with Lite-GPUs?](https://mlsanity.com/paper/2501.10187/) (2025)
- [One Search Fits All: Pareto-Optimal Eco-Friendly Model Selection](https://mlsanity.com/paper/2505.01468/) (2025)
- [Active Inference AI Systems for Scientific Discovery](https://mlsanity.com/paper/2506.21329/) (2025)
- [Nalar: An agent serving framework](https://mlsanity.com/paper/2601.05109/) (2026)
- [ViBE: Co-Optimizing Workload Skew and Hardware Variability for MoE Serving](https://mlsanity.com/paper/2606.00735/) (2026)
- [Magneton: Optimizing Energy Efficiency of ML Systems via Differential Energy Debugging](https://mlsanity.com/paper/2512.08365/) (2025)
- [Optimising for Energy Efficiency and Performance in Machine Learning](https://mlsanity.com/paper/2601.08991/) (2026)
- [Designing Datacenter Power Delivery Hierarchies for the AI Era](https://mlsanity.com/paper/2605.16255/) (2026)
- [GEM: GPU-Variability-Aware Expert to GPU Mapping for MoE Systems](https://mlsanity.com/paper/2605.19945/) (2026)
- [Evaluation of ML Resource Utilization Requires Model Life Cycle Assessment](https://mlsanity.com/paper/2606.07632/) (2026)
- [Efficient Resource Optimization for Split Federated Learning](https://mlsanity.com/paper/2608.17849/) (2026)
- [Phase-Decoupled, Model-Calibrated Power Control for Disaggregated LLM Serving](https://mlsanity.com/paper/2609.11133/) (2026)
- [Characterizing Job Power Elasticity for Power-Flexible AI Training](https://mlsanity.com/paper/2609.11542/) (2026)
- [Minibatch persistency, eight years later: what batch reuse costs in steps and joules, and what it saves in data](https://mlsanity.com/paper/2609.13922/) (2026)
- [A principled approach for energy-efficient training via phase-aware GPU frequency tuning](https://mlsanity.com/paper/2609.24205/) (2026)

## Abstract (arXiv)

Training large AI models on numerous GPUs consumes a massive amount of energy, making power delivery one of the largest limiting factors in building and operating datacenters for AI workloads. However, we observe that not all energy consumed during training directly contributes to end-to-end throughput; a significant portion can be removed without slowing down training. We call this portion energy bloat. In this work, we identify two independent sources of energy bloat in large model training and propose Perseus, a training system that mitigates both. To do this, Perseus obtains the time--energy tradeoff frontier of a large model training job using an efficient graph cut-based algorithm, and schedules computation energy consumption across time to reduce both types of energy bloat. Evaluation on large models, including GPT-3 and Bloom, shows that Perseus reduces the energy consumption of large model training by up to 30% without any throughput loss or hardware modification.