mlsanityData to 14 Oct 2026

Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model

arXiv 2211.02001 ↗First posted 3 Nov 2022cs.LGcited by 265

Not on the map. Why?

Abstract (arXiv)

Progress in machine learning (ML) comes with a cost to the environment, given that training ML models requires significant computational resources, energy and materials. In the present article, we aim to quantify the carbon footprint of BLOOM, a 176-billion parameter language model, across its life cycle. We estimate that BLOOM's final training emitted approximately 24.7 tonnes of~carboneq~if we consider only the dynamic power consumption, and 50.5 tonnes if we account for all processes ranging from equipment manufacturing to energy-based operational consumption. We also study the energy requirements and carbon emissions of its deployment for inference via an API endpoint receiving user queries in real-time. We conclude with a discussion regarding the difficulty of precisely estimating the carbon footprint of ML models and future research directions that can contribute towards improving carbon emissions reporting.

Key facts. Cited by 265 papers in the map (data through 14 Oct 2026). Builds on 14 of its 41 references that are in the map. First posted on arXiv 3 Nov 2022.

Builds on

14 of 41 references.

References · 14

  1. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  2. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,254
  3. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,202
  4. 2021Carbon Emissions and Large Neural Network Trainingcited by 567
  5. 2019Quantifying the Carbon Emissions of Machine Learningcited by 521
  6. 2021Sustainable AI: Environmental Implications, Challenges and Opportunitiescited by 242
  7. 2020The Computational Limits of Deep Learningcited by 193
  8. 2022The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrinkcited by 184
  9. 2020Carbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Modelscited by 169
  10. 2022What Language Model to Train if You Have One Million GPU Hours?cited by 92
  11. 2020Chasing Carbon: The Elusive Environmental Footprint of Computingcited by 72
  12. 2021Unraveling the Hidden Environmental Impacts of AI Solutions for Environmentcited by 32
2 more
  1. 2021Towards Quantifying the Carbon Emissions of Differentially Private Machine Learningcited by 8
  2. 2020SECure: A Social and Environmental Certificate for AI Systemscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 39; 2024: 74; 2025: 88; 2026: 62.

265 papers.

Sort:

By area:

  1. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  2. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
  3. 2024OLMo: Accelerating the Science of Language Modelscited by 613
  4. 2024BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domainscited by 301
  5. 2023A Survey on Model Compression for Large Language Modelscited by 269
  6. 2022Demystifying Prompts in Language Models via Perplexity Estimationcited by 216
  7. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 205
  8. 2023Power Hungry Processing: Watts Driving the Cost of AI Deployment?cited by 191
  9. 2023Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsitycited by 165
  10. 2024Machine Unlearning of Pre-trained Large Language Modelscited by 119
  11. 2024How to Train Data-Efficient LLMscited by 93
  12. 2023Evaluating the Social Impact of Generative AI Systems in Systems and Societycited by 92
38 more
  1. 2024Investigating Continual Pretraining in Large Language Models: Insights and Implicationscited by 57
  2. 2023Opening up ChatGPT: Tracking openness, transparency, and accountability in instruction-tuned text generatorscited by 56
  3. 2025Energy Considerations of Large Language Model Inference and Efficiency Optimizationscited by 55
  4. 2024CroissantLLM: A Truly Bilingual French-English Language Modelcited by 53
  5. 2025How Linguistics Learned to Stop Worrying and Love the Language Modelscited by 51
  6. 2023Post Hoc Explanations of Language Models Can Improve Language Modelscited by 50
  7. 2023Making AI Less "Thirsty": Uncovering and Addressing the Secret Water Footprint of AI Modelscited by 44
  8. 2023Counting Carbon: A Survey of Factors Influencing the Emissions of Machine Learningcited by 42
  9. 2024LQER: Low-Rank Quantization Error Reconstruction for LLMscited by 41
  10. 2024Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for Large Language Modelscited by 40
  11. 2024Memory3: Language Modeling with Explicit Memorycited by 40
  12. 2024Neural Methods for Amortized Inferencecited by 39
  13. 2023How to estimate carbon footprint when training deep learning models? A guide and reviewcited by 34
  14. 2025The ML.ENERGY Benchmark: Toward Automated Inference Energy Measurement and Optimizationcited by 33
  15. 2025Measuring the environmental impact of delivering AI at Google Scalecited by 32
  16. 2024The largest EEG-based BCI reproducibility study for open science: the MOABB benchmarkcited by 29
  17. 2023ChatGPT Needs SPADE (Sustainability, PrivAcy, Digital divide, and Ethics) Evaluation: A Reviewcited by 28
  18. 2024Offline Energy-Optimal LLM Serving: Workload-Based Energy Models for LLM Inference on Heterogeneous Systemscited by 28
  19. 2023DrBERT: A Robust Pre-trained Model in French for Biomedical and Clinical domainscited by 27
  20. 2024InfiniPot: Infinite Context Processing on Memory-Constrained LLMscited by 27
  21. 2025Holistically Evaluating the Environmental Impact of Creating Language Modelscited by 26
  22. 2023SIESTA: Efficient Online Continual Learning with Sleepcited by 24
  23. 2023How Efficient Are Today's Continual Learning Algorithms?cited by 24
  24. 2024The Price of Prompting: Profiling Energy Use in Large Language Models Inferencecited by 23
  25. 2023Reducing Energy Bloat in Large Model Trainingcited by 22
  26. 2025TAID: Temporally Adaptive Interpolated Distillation for Efficient Knowledge Transfer in Language Modelscited by 22
  27. 2024Holistic Safety and Responsibility Evaluations of Advanced AI Modelscited by 19
  28. 2024A Survey of Sustainability in Large Language Models: Applications, Economics, and Challengescited by 18
  29. 2023Unit Scaling: Out-of-the-Box Low-Precision Trainingcited by 17
  30. 2024Hybrid Heterogeneous Clusters Can Lower the Energy Consumption of LLM Inference Workloadscited by 17
  31. 2023An Interdisciplinary Outlook on Large Language Models for Scientific Researchcited by 15
  32. 2024CigaR: Cost-efficient Program Repair with LLMscited by 15
  33. 2024Enabling High-Sparsity Foundational Llama Models with Efficient Pretraining and Deploymentcited by 15
  34. 2024Foundation Model Transparency Reportscited by 14
  35. 2023Overcoming the Stability Gap in Continual Learningcited by 13
  36. 2024AI Art is Theft: Labour, Extraction, and Exploitation, Or, On the Dangers of Stochastic Pollockscited by 13
  37. 2024TeenyTinyLlama: open-source tiny language models trained in Brazilian Portuguesecited by 13
  38. 2024Toward Sustainable GenAI using Generation Directives for Carbon-Friendly Large Language Model Inferencecited by 13

50 most cited of 265 All 265 (JSON)