# Green AI: Exploring Carbon Footprints, Mitigation Strategies, and Trade Offs in Large Language Model Training arXiv: https://arxiv.org/abs/2404.01157 ยท HTML: https://mlsanity.com/paper/2404.01157/ ## Key facts - Cited by 15 papers in the map (data through 14 Oct 2026). - Builds on 18 of its 25 references that are in the map. - First posted on arXiv 1 Apr 2024. ## Builds on (most cited first, 18 of 18) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018) - [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020) - [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019) - [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019) - [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019) - [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016) - [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019) - [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019) - [What Does BERT Look At? An Analysis of BERT's Attention](https://mlsanity.com/paper/1906.04341/) (2019) - [Green AI](https://mlsanity.com/paper/1907.10597/) (2019) - [Quantifying the Carbon Emissions of Machine Learning](https://mlsanity.com/paper/1910.09700/) (2019) - [Pretrained Transformers for Text Ranking: BERT and Beyond](https://mlsanity.com/paper/2010.06467/) (2020) - [Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehension](https://mlsanity.com/paper/2002.00293/) (2020) - [Eco2AI: carbon emissions tracking of machine learning models as the first step towards sustainable AI](https://mlsanity.com/paper/2208.00406/) (2022) - [Counting Carbon: A Survey of Factors Influencing the Emissions of Machine Learning](https://mlsanity.com/paper/2302.08476/) (2023) - [ChatGPT for Shaping the Future of Dentistry: The Potential of Multi-Modal Large Language Model](https://mlsanity.com/paper/2304.03086/) (2023) - [Chatbot Interaction with Artificial Intelligence: Human Data Augmentation with T5 and Language Transformer Ensemble for Text Classification](https://mlsanity.com/paper/2010.05990/) (2020) ## Built on by (most cited first, 15 of 15) - [On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectification](https://mlsanity.com/paper/2508.05629/) (2025) - [The Rise of Small Language Models in Healthcare: A Comprehensive Survey](https://mlsanity.com/paper/2504.17119/) (2025) - [Risks, Causes, and Mitigations of Widespread Deployments of Large Language Models (LLMs): A Survey](https://mlsanity.com/paper/2408.04643/) (2024) - [Watt Counts: Energy-Aware Benchmark for Sustainable LLM Inference on Heterogeneous GPU Architectures](https://mlsanity.com/paper/2604.09048/) (2026) - [Optimizing Large Language Models: Metrics, Energy Efficiency, and Case Study Insights](https://mlsanity.com/paper/2504.06307/) (2025) - [SLM-Bench: A Comprehensive Benchmark of Small Language Models on Environmental Impacts--Extended Version](https://mlsanity.com/paper/2508.15478/) (2025) - [RETUYT-INCO at BEA 2025 Shared Task: How Far Can Lightweight Models Go in AI-powered Tutor Evaluation?](https://mlsanity.com/paper/2506.11243/) (2025) - [Hidden State Poisoning Attacks against Mamba-based Language Models](https://mlsanity.com/paper/2601.01972/) (2026) - [FairSense-AI: Responsible AI Meets Sustainability](https://mlsanity.com/paper/2503.02865/) (2025) - [AOLO: Analysis and Optimization For Low-Carbon Oriented Wireless Large Language Model Services](https://mlsanity.com/paper/2503.04418/) (2025) - [When Discourse Stalls: Moving Past Five Semantic Stopsigns about Generative AI in Design Research](https://mlsanity.com/paper/2503.08565/) (2025) - [CLASP: Defending Hybrid Large Language Models Against Hidden State Poisoning Attacks](https://mlsanity.com/paper/2603.12206/) (2026) - [Unifying Data, Memory, and Compute Efficiency in LLM training: A Survey](https://mlsanity.com/paper/2606.10706/) (2026) - [Decision Protocols in Multi-Agent Large Language Model Conversations](https://mlsanity.com/paper/2607.05477/) (2026) - [LAAF: A Layered Accountability Architecture Framework for LLM Applications](https://mlsanity.com/paper/2608.27102/) (2026) ## Abstract (arXiv) Prominent works in the field of Natural Language Processing have long attempted to create new innovative models by improving upon previous model training approaches, altering model architecture, and developing more in-depth datasets to better their performance. However, with the quickly advancing field of NLP comes increased greenhouse gas emissions, posing concerns over the environmental damage caused by training LLMs. Gaining a comprehensive understanding of the various costs, particularly those pertaining to environmental aspects, that are associated with artificial intelligence serves as the foundational basis for ensuring safe AI models. Currently, investigations into the CO2 emissions of AI models remain an emerging area of research, and as such, in this paper, we evaluate the CO2 emissions of well-known large language models, which have an especially high carbon footprint due to their significant amount of model parameters. We argue for the training of LLMs in a way that is responsible and sustainable by suggesting measures for reducing carbon emissions. Furthermore, we discuss how the choice of hardware affects CO2 emissions by contrasting the CO2 emissions during model training for two widely used GPUs. Based on our results, we present the benefits and drawbacks of our proposed solutions and make the argument for the possibility of training more environmentally safe AI models without sacrificing their robustness and performance.