mlsanityData to 7 Oct 2026

Beyond Efficiency: Scaling AI Sustainably

arXiv 2406.05303 ↗First posted 8 Jun 2024cs.LG cs.DCcited by 8

Not on the map. Why?

Abstract (arXiv)

Barroso's seminal contributions in energy-proportional warehouse-scale computing launched an era where modern datacenters have become more energy efficient and cost effective than ever before. At the same time, modern AI applications have driven ever-increasing demands in computing, highlighting the importance of optimizing efficiency across the entire deep learning model development cycle. This paper characterizes the carbon impact of AI, including both operational carbon emissions from training and inference as well as embodied carbon emissions from datacenter construction and hardware manufacturing. We highlight key efficiency optimization opportunities for cutting-edge AI technologies, from deep learning recommendation models to multi-modal generative AI tasks. To scale AI sustainably, we must also go beyond efficiency and optimize across the life cycle of computing infrastructures, from hardware manufacturing to datacenter operations and end-of-life processing for the hardware.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 30 of its 66 references that are in the map. First posted on arXiv 8 Jun 2024.

Builds on

30 of 66 references.

References · 30

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2020Scaling Laws for Neural Language Modelscited by 6,411
  5. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  6. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  7. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  8. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  9. 2021Carbon Emissions and Large Neural Network Trainingcited by 561
  10. 2022Beyond neural scaling laws: beating power law scaling via data pruningcited by 519
  11. 2022Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Modelcited by 264
  12. 2020Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learningcited by 246
18 more
  1. 2021Sustainable AI: Environmental Implications, Challenges and Opportunitiescited by 242
  2. 2022The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrinkcited by 183
  3. 2019The Architectural Implications of Facebook's DNN-based Personalized Recommendationcited by 83
  4. 2020Chasing Carbon: The Elusive Environmental Footprint of Computingcited by 71
  5. 2020An Introduction to Electrocatalyst Design using Machine Learning for Renewable Energy Storagecited by 45
  6. 2021Carbon-Aware Computing for Datacenterscited by 40
  7. 2022Carbon Explorer: A Holistic Approach for Designing Carbon Aware Datacenterscited by 31
  8. 2021Understanding Data Storage and Ingestion for Large-Scale Deep Recommendation Model Trainingcited by 28
  9. 2021SVP-CF: Selection via Proxy for Collaborative Filtering Datacited by 14
  10. 2023MAD Max Beyond Single-Node: Enabling Large Machine Learning Model Acceleration on Distributed Systemscited by 14
  11. 2022Treehouse: A Case For Carbon-Aware Datacenter Softwarecited by 12
  12. 2019Summarizing CPU and GPU Design Trends with Product Datacited by 10
  13. 2023Adapting Datacenter Capacity for Greener Datacenters and Gridcited by 6
  14. 2023Generative AI Beyond LLMs: System Implications of Multi-Modal Generationcited by 5
  15. 2023Carbon Responder: Coordinating Demand Response for the Datacenter Fleetcited by 4
  16. 2019Accelerator-level Parallelismcited by 2
  17. 2021Socio-Technological Challenges and Opportunities: Paths Forwardcited by 2
  18. 2024Unlocking the Potential of Renewable Energy Through Curtailment Predictioncited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 3; 2026: 2.

8 papers.

  1. 2024Hardware Scaling Trends and Diminishing Returns in Large-Scale Distributed Trainingcited by 7
  2. 2025Position: AI Scaling: From Up to Down and Outcited by 5
  3. 2024The Energy Cost of Artificial Intelligence Lifecycle in Communication Networkscited by 4
  4. 2025Scaling Laws Meet Model Architecture: Toward Inference-Efficient LLMscited by 4
  5. 2024Characterizing and Efficiently Accelerating Multimodal Generation Model Inferencecited by 1
  6. 2025Breaking the ICE: Exploring promises and challenges of benchmarks for Inference Carbon & Energy estimation for LLMscited by 1
  7. 2026Artificial Intelligence in Open Source Software Engineering: A Foundation for Sustainabilitycited by 1
  8. 2026Artificial Intelligence for Energy Optimization in Data Centerscited by 0