# Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model

arXiv: https://arxiv.org/abs/2211.02001 · HTML: https://mlsanity.com/paper/2211.02001/

## Key facts

- Cited by 265 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 41 references that are in the map.
- First posted on arXiv 3 Nov 2022.

## Builds on (most cited first, 14 of 14)

- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Scaling Language Models: Methods, Analysis & Insights from Training Gopher](https://mlsanity.com/paper/2112.11446/) (2021)
- [Carbon Emissions and Large Neural Network Training](https://mlsanity.com/paper/2104.10350/) (2021)
- [Quantifying the Carbon Emissions of Machine Learning](https://mlsanity.com/paper/1910.09700/) (2019)
- [Sustainable AI: Environmental Implications, Challenges and Opportunities](https://mlsanity.com/paper/2111.00364/) (2021)
- [The Computational Limits of Deep Learning](https://mlsanity.com/paper/2007.05558/) (2020)
- [The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink](https://mlsanity.com/paper/2204.05149/) (2022)
- [Carbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Models](https://mlsanity.com/paper/2007.03051/) (2020)
- [What Language Model to Train if You Have One Million GPU Hours?](https://mlsanity.com/paper/2210.15424/) (2022)
- [Chasing Carbon: The Elusive Environmental Footprint of Computing](https://mlsanity.com/paper/2011.02839/) (2020)
- [Unraveling the Hidden Environmental Impacts of AI Solutions for Environment](https://mlsanity.com/paper/2110.11822/) (2021)
- [Towards Quantifying the Carbon Emissions of Differentially Private Machine Learning](https://mlsanity.com/paper/2107.06946/) (2021)
- [SECure: A Social and Environmental Certificate for AI Systems](https://mlsanity.com/paper/2006.06217/) (2020)

## Built on by (most cited first, 30 of 265)

- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis](https://mlsanity.com/paper/2310.00426/) (2023)
- [OLMo: Accelerating the Science of Language Models](https://mlsanity.com/paper/2402.00838/) (2024)
- [BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domains](https://mlsanity.com/paper/2402.10373/) (2024)
- [A Survey on Model Compression for Large Language Models](https://mlsanity.com/paper/2308.07633/) (2023)
- [Demystifying Prompts in Language Models via Perplexity Estimation](https://mlsanity.com/paper/2212.04037/) (2022)
- [Large Language Model Agent: A Survey on Methodology, Applications and Challenges](https://mlsanity.com/paper/2503.21460/) (2025)
- [Power Hungry Processing: Watts Driving the Cost of AI Deployment?](https://mlsanity.com/paper/2311.16863/) (2023)
- [Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity](https://mlsanity.com/paper/2310.05175/) (2023)
- [Machine Unlearning of Pre-trained Large Language Models](https://mlsanity.com/paper/2402.15159/) (2024)
- [How to Train Data-Efficient LLMs](https://mlsanity.com/paper/2402.09668/) (2024)
- [Evaluating the Social Impact of Generative AI Systems in Systems and Society](https://mlsanity.com/paper/2306.05949/) (2023)
- [Investigating Continual Pretraining in Large Language Models: Insights and Implications](https://mlsanity.com/paper/2402.17400/) (2024)
- [Opening up ChatGPT: Tracking openness, transparency, and accountability in instruction-tuned text generators](https://mlsanity.com/paper/2307.05532/) (2023)
- [Energy Considerations of Large Language Model Inference and Efficiency Optimizations](https://mlsanity.com/paper/2504.17674/) (2025)
- [CroissantLLM: A Truly Bilingual French-English Language Model](https://mlsanity.com/paper/2402.00786/) (2024)
- [How Linguistics Learned to Stop Worrying and Love the Language Models](https://mlsanity.com/paper/2501.17047/) (2025)
- [Post Hoc Explanations of Language Models Can Improve Language Models](https://mlsanity.com/paper/2305.11426/) (2023)
- [Making AI Less "Thirsty": Uncovering and Addressing the Secret Water Footprint of AI Models](https://mlsanity.com/paper/2304.03271/) (2023)
- [Counting Carbon: A Survey of Factors Influencing the Emissions of Machine Learning](https://mlsanity.com/paper/2302.08476/) (2023)
- [LQER: Low-Rank Quantization Error Reconstruction for LLMs](https://mlsanity.com/paper/2402.02446/) (2024)
- [Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for Large Language Models](https://mlsanity.com/paper/2406.02924/) (2024)
- [Memory3: Language Modeling with Explicit Memory](https://mlsanity.com/paper/2407.01178/) (2024)
- [Neural Methods for Amortized Inference](https://mlsanity.com/paper/2404.12484/) (2024)
- [How to estimate carbon footprint when training deep learning models? A guide and review](https://mlsanity.com/paper/2306.08323/) (2023)
- [The ML.ENERGY Benchmark: Toward Automated Inference Energy Measurement and Optimization](https://mlsanity.com/paper/2505.06371/) (2025)
- [Measuring the environmental impact of delivering AI at Google Scale](https://mlsanity.com/paper/2508.15734/) (2025)
- [The largest EEG-based BCI reproducibility study for open science: the MOABB benchmark](https://mlsanity.com/paper/2404.15319/) (2024)
- [ChatGPT Needs SPADE (Sustainability, PrivAcy, Digital divide, and Ethics) Evaluation: A Review](https://mlsanity.com/paper/2305.03123/) (2023)
- [Offline Energy-Optimal LLM Serving: Workload-Based Energy Models for LLM Inference on Heterogeneous Systems](https://mlsanity.com/paper/2407.04014/) (2024)

## Abstract (arXiv)

Progress in machine learning (ML) comes with a cost to the environment, given that training ML models requires significant computational resources, energy and materials. In the present article, we aim to quantify the carbon footprint of BLOOM, a 176-billion parameter language model, across its life cycle. We estimate that BLOOM's final training emitted approximately 24.7 tonnes of~carboneq~if we consider only the dynamic power consumption, and 50.5 tonnes if we account for all processes ranging from equipment manufacturing to energy-based operational consumption. We also study the energy requirements and carbon emissions of its deployment for inference via an API endpoint receiving user queries in real-time. We conclude with a discussion regarding the difficulty of precisely estimating the carbon footprint of ML models and future research directions that can contribute towards improving carbon emissions reporting.