mlsanityData to 7 Oct 2026

Sustainable AI: Environmental Implications, Challenges and Opportunities

arXiv 2111.00364 ↗First posted 30 Oct 2021cs.LG cs.AI cs.ARcited by 242

Not on the map. Why?

Abstract (arXiv)

This paper explores the environmental impact of the super-linear growth trends for AI from a holistic perspective, spanning Data, Algorithms, and System Hardware. We characterize the carbon footprint of AI computing by examining the model development cycle across industry-scale machine learning use cases and, at the same time, considering the life cycle of system hardware. Taking a step further, we capture the operational and manufacturing carbon footprint of AI computing and present an end-to-end analysis for what and how hardware-software design and at-scale optimization can help reduce the overall carbon footprint of AI. Based on the industry experience and lessons learned, we share the key challenges and chart out important development directions across the many dimensions of AI. We hope the key messages and insights presented in this paper can inspire the community to advance the field of AI in an environmentally-responsible manner.

Key facts. Cited by 242 papers in the map (data through 7 Oct 2026). Builds on 68 of its 128 references that are in the map. First posted on arXiv 30 Oct 2021.

Builds on

68 of 128 references.

References · 68

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  4. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  5. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  6. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  7. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  8. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  9. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  10. 2018Model Cards for Model Reportingcited by 1,068
  11. 2019Towards Federated Learning at Scale: System Designcited by 826
  12. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 778
38 more
  1. 2019Green AIcited by 620
  2. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 610
  3. 2018Federated Learning for Mobile Keyboard Predictioncited by 565
  4. 2021Carbon Emissions and Large Neural Network Trainingcited by 561
  5. 2019Quantifying the Carbon Emissions of Machine Learningcited by 516
  6. 2021Dynabench: Rethinking Benchmarking in NLPcited by 447
  7. 2019Deep Learning Recommendation Model for Personalization and Recommendation Systemscited by 327
  8. 2019Tackling Climate Change with Machine Learningcited by 264
  9. 2020A Comprehensive Survey of Neural Architecture Search: Challenges and Solutionscited by 253
  10. 2021ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learningcited by 253
  11. 2020Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learningcited by 246
  12. 2020Neural Architecture Search without Trainingcited by 241
  13. 2019PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimizationcited by 234
  14. 2021Internet-Augmented Dialogue Generationcited by 217
  15. 2018Scaling Neural Machine Translationcited by 204
  16. 2018Applied Federated Learning: Improving Google Keyboard Query Suggestionscited by 202
  17. 2019MLPerf Inference Benchmarkcited by 195
  18. 2021Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspectivecited by 168
  19. 2019Federated Evaluation of On-device Personalizationcited by 147
  20. 2021Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020cited by 146
  21. 2016Designing Energy-Efficient Convolutional Neural Networks using Energy-Aware Pruningcited by 133
  22. 2019Federated Learning for Emoji Prediction in a Mobile Keyboardcited by 115
  23. 2019MLPerf Training Benchmarkcited by 114
  24. 2021Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with Support Samplescited by 114
  25. 2019On Empirical Comparisons of Optimizers for Deep Learningcited by 99
  26. 2019Single-Path NAS: Designing Hardware-Efficient ConvNets in less than 4 Hourscited by 87
  27. 2019The Architectural Implications of Facebook's DNN-based Personalized Recommendationcited by 83
  28. 2019Analysis of Large-Scale Multi-Tenant GPU Clusters for DNN Training Workloadscited by 79
  29. 2021Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Modelscited by 78
  30. 2021On Large-Cohort Training for Federated Learningcited by 73
  31. 2020Chasing Carbon: The Elusive Environmental Footprint of Computingcited by 71
  32. 2020Distributed Hierarchical GPU Parameter Server for Massive Scale Deep Learning Ads Systemscited by 59
  33. 2021TT-Rec: Tensor Train Compression for Deep Learning Recommendation Modelscited by 58
  34. 2020Can Federated Learning Save The Planet?cited by 52
  35. 2020Measuring the Algorithmic Efficiency of Neural Networkscited by 51
  36. 2019RecNMP: Accelerating Personalized Recommendation with Near-Memory Processingcited by 49
  37. 2020An Introduction to Electrocatalyst Design using Machine Learning for Renewable Energy Storagecited by 45
  38. 2019Energy Usage Reports: Environmental awareness as part of algorithmic accountabilitycited by 43

50 most cited of 68 All 68 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 24; 2023: 68; 2024: 59; 2025: 45; 2026: 44.

242 papers.

Sort:

By area:

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  4. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 791
  5. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  6. 2022Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Modelcited by 264
  7. 2023Power Hungry Processing: Watts Driving the Cost of AI Deployment?cited by 186
  8. 2021Efficient Large Scale Language Modeling with Mixtures of Expertscited by 175
  9. 2022MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoningcited by 153
  10. 2023Large language models can accurately predict searcher preferencescited by 129
  11. 2022Measuring the Carbon Intensity of AI in Cloud Instancescited by 110
  12. 2023Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inferencecited by 101
38 more
  1. 2023Algorithm of Thoughts: Enhancing Exploration of Ideas in Large Language Modelscited by 92
  2. 2023Evaluating the Social Impact of Generative AI Systems in Systems and Societycited by 91
  3. 2023Data Distillation: A Surveycited by 86
  4. 2023Red teaming ChatGPT via Jailbreaking: Bias, Robustness, Reliability and Toxicitycited by 86
  5. 2024A Survey of Resource-efficient LLM and Multimodal Foundation Modelscited by 84
  6. 2022Zeus: Understanding and Optimizing GPU Energy Consumption of DNN Trainingcited by 78
  7. 2022Efficient Methods for Natural Language Processing: A Surveycited by 78
  8. 2023A Systematic Review of Green AIcited by 77
  9. 2024Foundation Model for Advancing Healthcare: Challenges, Opportunities, and Future Directionscited by 77
  10. 2023LLMCarbon: Modeling the end-to-end Carbon Footprint of Large Language Modelscited by 71
  11. 2023E^2VPT: An Effective and Efficient Approach for Visual Prompt Tuningcited by 70
  12. 2023Connecting the Dots in Trustworthy Artificial Intelligence: From AI Principles, Ethics, and Key Requirements to Responsible AI Systems and Regulationcited by 60
  13. 2023Making AI Less "Thirsty": Uncovering and Addressing the Secret Water Footprint of AI Modelscited by 44
  14. 2023Counting Carbon: A Survey of Factors Influencing the Emissions of Machine Learningcited by 42
  15. 2024Evolving Code with A Large Language Modelcited by 42
  16. 2024When Large Language Models Meet Vector Databases: A Surveycited by 39
  17. 2022Understanding Scaling Laws for Recommendation Modelscited by 38
  18. 2024MELTing point: Mobile Evaluation of Language Transformerscited by 35
  19. 2023How to estimate carbon footprint when training deep learning models? A guide and reviewcited by 34
  20. 2024On the Challenges and Opportunities in Generative AIcited by 31
  21. 2022Green Learning: Introduction, Examples and Outlookcited by 30
  22. 2022RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendationcited by 28
  23. 2024ICONS: Influence Consensus for Vision-Language Data Selectioncited by 28
  24. 2024Offline Energy-Optimal LLM Serving: Workload-Based Energy Models for LLM Inference on Heterogeneous Systemscited by 27
  25. 2025Holistically Evaluating the Environmental Impact of Creating Language Modelscited by 26
  26. 2022Data-Centric Green AI: An Exploratory Empirical Studycited by 24
  27. 2023SIESTA: Efficient Online Continual Learning with Sleepcited by 24
  28. 2024Towards Cognitive AI Systems: a Survey and Prospective on Neuro-Symbolic AIcited by 24
  29. 2024MoralBERT: A Fine-Tuned Language Model for Capturing Moral Values in Social Discussionscited by 23
  30. 2022Training Overparametrized Neural Networks in Sublinear Timecited by 22
  31. 2023An Overview on Language Models: Recent Developments and Outlookcited by 22
  32. 2023High-performance deep spiking neural networks with 0.3 spikes per neuroncited by 22
  33. 2022Towards Climate Awareness in NLP Researchcited by 19
  34. 2023AI-native Interconnect Framework for Integration of Large Language Model Technologies in 6G Systemscited by 19
  35. 2024A Survey of Optimization-based Task and Motion Planning: From Classical To Learning Approachescited by 19
  36. 2023FLINT: A Platform for Federated Learning Integrationcited by 17
  37. 2023A Synthesis of Green Architectural Tactics for ML-Enabled Systemscited by 17
  38. 2024Hybrid Heterogeneous Clusters Can Lower the Energy Consumption of LLM Inference Workloadscited by 17

50 most cited of 242 All 242 (JSON)