mlsanityData to 7 Oct 2026

The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink

arXiv 2204.05149 ↗First posted 11 Apr 2022cs.LG cs.AI cs.GLcited by 183

Not on the map. Why?

Abstract (arXiv)

Machine Learning (ML) workloads have rapidly grown in importance, but raised concerns about their carbon footprint. Four best practices can reduce ML training energy by up to 100x and CO2 emissions up to 1000x. By following best practices, overall ML energy use (across research, development, and production) held steady at <15% of Google's total energy use for the past three years. If the whole ML field were to adopt best practices, total carbon emissions from training would reduce. Hence, we recommend that ML papers include emissions explicitly to foster competition on more than just model quality. Estimates of emissions in papers that omitted them have been off 100x-100,000x, so publishing emissions has the added benefit of ensuring accurate accounting. Given the importance of climate change, we must get the numbers right to make certain that we work on its biggest challenges.

Key facts. Cited by 183 papers in the map (data through 7 Oct 2026). Builds on 11 of its 31 references that are in the map. First posted on arXiv 11 Apr 2022.

Builds on

11 of 31 references.

References · 11

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  3. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  4. 2019Green AIcited by 620
  5. 2021Carbon Emissions and Large Neural Network Trainingcited by 561
  6. 2019Quantifying the Carbon Emissions of Machine Learningcited by 516
  7. 2020Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learningcited by 246
  8. 2020The Computational Limits of Deep Learningcited by 188
  9. 2019The Evolved Transformercited by 180
  10. 2021Primer: Searching for Efficient Transformers for Language Modelingcited by 162
  11. 2021Searching for Fast Model Families on Datacenter Acceleratorscited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 16; 2023: 42; 2024: 31; 2025: 52; 2026: 42.

183 papers.

Sort:

By area:

  1. 2022Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Modelcited by 264
  2. 2023TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddingscited by 198
  3. 2023Power Hungry Processing: Watts Driving the Cost of AI Deployment?cited by 186
  4. 2023Large language models can accurately predict searcher preferencescited by 129
  5. 2022Measuring the Carbon Intensity of AI in Cloud Instancescited by 110
  6. 2025OverThink: Slowdown Attacks on Reasoning LLMscited by 94
  7. 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 91
  8. 2023A Systematic Review of Green AIcited by 77
  9. 2023LLMCarbon: Modeling the end-to-end Carbon Footprint of Large Language Modelscited by 71
  10. 2023Large AI Models in Health Informatics: Applications, Challenges, and the Futurecited by 61
  11. 2023Connecting the Dots in Trustworthy Artificial Intelligence: From AI Principles, Ethics, and Key Requirements to Responsible AI Systems and Regulationcited by 60
  12. 2025Energy Considerations of Large Language Model Inference and Efficiency Optimizationscited by 54
38 more
  1. 2022On the Principles of Parsimony and Self-Consistency for the Emergence of Intelligencecited by 50
  2. 2023Making AI Less "Thirsty": Uncovering and Addressing the Secret Water Footprint of AI Modelscited by 44
  3. 2023Counting Carbon: A Survey of Factors Influencing the Emissions of Machine Learningcited by 42
  4. 2023No Train No Gain: Revisiting Efficient Training Algorithms For Transformer-based Language Modelscited by 41
  5. 2022Compute-Efficient Deep Learning: Algorithmic Trends and Opportunitiescited by 38
  6. 2024MELTing point: Mobile Evaluation of Language Transformerscited by 35
  7. 2024Crabs: Consuming Resource via Auto-generation for LLM-DoS Attack under Black-box Settingscited by 33
  8. 2025Measuring the environmental impact of delivering AI at Google Scalecited by 31
  9. 2022What Do NLP Researchers Believe? Results of the NLP Community Metasurveycited by 28
  10. 2023ChatGPT Needs SPADE (Sustainability, PrivAcy, Digital divide, and Ethics) Evaluation: A Reviewcited by 28
  11. 2025Entropy-based Exploration Conduction for Multi-step Reasoningcited by 28
  12. 2022POET: Training Neural Networks on Tiny Devices with Integrated Rematerialization and Pagingcited by 26
  13. 2025Jailbreaking Large Language Diffusion Models: Revealing Hidden Safety Flaws in Diffusion-Based Text Generationcited by 24
  14. 2023Exploring the Carbon Footprint of Hugging Face's ML Models: A Repository Mining Studycited by 23
  15. 2023High-performance deep spiking neural networks with 0.3 spikes per neuroncited by 22
  16. 2025AnnaAgent: Dynamic Evolution Agent System with Multi-Session Memory for Realistic Seeker Simulationcited by 21
  17. 2025DynTaskMAS: A Dynamic Task Graph-driven Framework for Asynchronous and Parallel LLM-based Multi-Agent Systemscited by 20
  18. 2022Towards Climate Awareness in NLP Researchcited by 19
  19. 2023AI-native Interconnect Framework for Integration of Large Language Model Technologies in 6G Systemscited by 19
  20. 2025ModernVBERT: Towards Smaller Visual Document Retrieverscited by 19
  21. 2023TinyTrain: Resource-Aware Task-Adaptive Sparse Training of DNNs at the Data-Scarce Edgecited by 18
  22. 2024SLO-aware GPU Frequency Scaling for Energy Efficient LLM Inference Servingcited by 18
  23. 2025GTPO: Stabilizing Group Relative Policy Optimization via Gradient and Entropy Controlcited by 18
  24. 2025Can LLM Agents Maintain a Persona in Discourse?cited by 15
  25. 2023MAD Max Beyond Single-Node: Enabling Large Machine Learning Model Acceleration on Distributed Systemscited by 14
  26. 2025The Energy Cost of Reasoning: Analyzing Energy Usage in LLMs with Test-time Computecited by 14
  27. 2024Toward Sustainable GenAI using Generation Directives for Carbon-Friendly Large Language Model Inferencecited by 13
  28. 2025Learning to Focus: Causal Attention Distillation via Gradient-Guided Token Pruningcited by 13
  29. 2025Promptception: How Sensitive Are Large Multimodal Models to Prompts?cited by 13
  30. 2022The Future of Consumer Edge-AI Computingcited by 12
  31. 2023Efficiency is Not Enough: A Critical Perspective of Environmentally Sustainable AIcited by 12
  32. 2024Lessons from the Use of Natural Language Inference (NLI) in Requirements Engineering Taskscited by 12
  33. 2024Tucano: Advancing Neural Text Generation for Portuguesecited by 12
  34. 2024Towards Sustainable Large Language Model Servingcited by 12
  35. 2023The Framework Tax: Disparities Between Inference Efficiency in NLP Research and Deploymentcited by 11
  36. 2023FedZero: Leveraging Renewable Excess Energy in Federated Learningcited by 11
  37. 2023Energy and Carbon Considerations of Fine-Tuning BERTcited by 11
  38. 2025Towards Sustainable NLP: Insights from Benchmarking Inference Energy in Large Language Modelscited by 11

50 most cited of 183 All 183 (JSON)