mlsanityData to 14 Oct 2026

Power Hungry Processing: Watts Driving the Cost of AI Deployment?

arXiv 2311.16863 ↗First posted 28 Nov 2023cs.LGcited by 191

Not on the map. Why?

Abstract (arXiv)

Recent years have seen a surge in the popularity of commercial AI products based on generative, multi-purpose AI systems promising a unified approach to building machine learning (ML) models into technology. However, this ambition of ``generality'' comes at a steep cost to the environment, given the amount of energy these systems require and the amount of carbon that they emit. In this work, we propose the first systematic comparison of the ongoing inference cost of various categories of ML systems, covering both task-specific (i.e. finetuned models that carry out a single task) and `general-purpose' models, (i.e. those trained for multiple tasks). We measure deployment cost as the amount of energy and carbon required to perform 1,000 inferences on representative benchmark dataset using these models. We find that multi-purpose, generative architectures are orders of magnitude more expensive than task-specific systems for a variety of tasks, even when controlling for the number of model parameters. We conclude with a discussion around the current trend of deploying multi-purpose generative ML systems, and caution that their utility should be more intentionally weighed against increased costs in terms of energy and emissions. All the data from our study can be accessed via an interactive demo to carry out further exploration and analysis.

Key facts. Cited by 191 papers in the map (data through 14 Oct 2026). Builds on 32 of its 60 references that are in the map. First posted on arXiv 28 Nov 2023.

Builds on

32 of 60 references.

References · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  7. 2016Pointer Sentinel Mixture Modelscited by 3,096
  8. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  9. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  10. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  11. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  12. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
20 more
  1. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  2. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,317
  3. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,254
  4. 2015Teaching Machines to Read and Comprehendcited by 1,049
  5. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,043
  6. 2017Crowdsourcing Multiple Choice Science Questionscited by 765
  7. 2021Carbon Emissions and Large Neural Network Trainingcited by 567
  8. 2019SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarizationcited by 535
  9. 2019Quantifying the Carbon Emissions of Machine Learningcited by 521
  10. 2022DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Modelscited by 379
  11. 2022Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Modelcited by 265
  12. 2021Sustainable AI: Environmental Implications, Challenges and Opportunitiescited by 242
  13. 2022The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrinkcited by 184
  14. 2021RedCaps: web-curated image-text data created by the people, for the peoplecited by 180
  15. 2022Measuring the Carbon Intensity of AI in Cloud Instancescited by 111
  16. 2021Compute and Energy Consumption Trends in Deep Learning Inferencecited by 61
  17. 2023Counting Carbon: A Survey of Factors Influencing the Emissions of Machine Learningcited by 42
  18. 2022Evaluate & Evaluation on the Hub: Better Best Practices for Data and Model Measurementscited by 22
  19. 2023CarbonScaler: Leveraging Cloud Workload Elasticity for Optimizing Carbon-Efficiencycited by 17
  20. 2021CPPE-5: Medical Personal Protective Equipment Datasetcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 33; 2025: 69; 2026: 86.

191 papers.

Sort:

By area:

  1. 2025OverThink: Slowdown Attacks on Reasoning LLMscited by 96
  2. 2024Multistep Distillation of Diffusion Models via Moment Matchingcited by 82
  3. 2025How Hungry is AI? Benchmarking Energy, Water, and Carbon Footprint of LLM Inferencecited by 76
  4. 2024Multistep Consistency Modelscited by 67
  5. 2024A Review of Modern Recommender Systems Using Generative Models (Gen-RecSys)cited by 58
  6. 2025Energy Considerations of Large Language Model Inference and Efficiency Optimizationscited by 55
  7. 2024Diffusion Models and Representation Learning: A Surveycited by 54
  8. 2025How Linguistics Learned to Stop Worrying and Love the Language Modelscited by 51
  9. 2025Fully Autonomous AI Agents Should Not be Developedcited by 45
  10. 2024Participation in the age of foundation modelscited by 32
  11. 2025Holistically Evaluating the Environmental Impact of Creating Language Modelscited by 26
  12. 2024The Price of Prompting: Profiling Energy Use in Large Language Models Inferencecited by 23
38 more
  1. 2025Virtual Agent Economiescited by 21
  2. 2025The Geometry of LLM Quantization: GPTQ as Babai's Nearest Plane Algorithmcited by 20
  3. 2024LLMCO2: Advancing Accurate Carbon Footprint Prediction for LLM Inferencescited by 19
  4. 2025Understanding the LLM-ification of CHI: Unpacking the Impact of LLMs at CHI through a Systematic Literature Reviewcited by 16
  5. 2025The Energy Cost of Reasoning: Analyzing Energy Usage in LLMs with Test-time Computecited by 15
  6. 2024Mask in the Mirror: Implicit Sparsificationcited by 14
  7. 2026Beyond Max Tokens: Stealthy Resource Amplification via Tool Calling Chains in LLM Agentscited by 14
  8. 2026Sustainable Hybrid Document-Routed Retrieval for Financial RAG: Resolving the Robustness-Precision Trade-offcited by 13
  9. 2023Evaluation is all you need. Prompting Generative Large Language Models for Annotation Tasks in the Social Sciences. A Primer using Open Modelscited by 12
  10. 2024Position: An Inner Interpretability Framework for AI Inspired by Lessons from Cognitive Neurosciencecited by 11
  11. 2025Towards Sustainable NLP: Insights from Benchmarking Inference Energy in Large Language Modelscited by 11
  12. 2025Toward Holistic Evaluation of Recommender Systems Powered by Generative Modelscited by 11
  13. 2023Vectorizing string entries for data processing on tables: when are larger language models better?cited by 10
  14. 2025A Survey of Scaling in Large Language Model Reasoningcited by 10
  15. 2025Does quantization affect models' performance on long-context tasks?cited by 10
  16. 2024Personalized Representation from Personalized Generationcited by 9
  17. 2025Brevity is the soul of sustainability: Characterizing LLM response lengthscited by 9
  18. 2024Hardware Scaling Trends and Diminishing Returns in Large-Scale Distributed Trainingcited by 8
  19. 2025Greening AI-enabled Systems with Software Engineering: A Research Agenda for Environmentally Sustainable AI Practicescited by 8
  20. 2025Ground-Truthing AI Energy Consumption: Validating CodeCarbon Against External Measurementscited by 8
  21. 2024Efficient Deep Learning with Decorrelated Backpropagationcited by 7
  22. 2025CATransformers: Carbon Aware Transformers Through Joint Model-Hardware Optimizationcited by 7
  23. 2024The Impact of Quantization and Pruning on Deep Reinforcement Learning Modelscited by 6
  24. 2024Shaking Up VLMs: Comparing Transformers and Structured State Space Models for Vision & Language Modelingcited by 6
  25. 2025LACONIC: A 3D Layout Adapter for Controllable Image Creationcited by 6
  26. 2025Advancing Dialectal Arabic to Modern Standard Arabic Machine Translationcited by 6
  27. 2023Diffusion Models for Generative Artificial Intelligence: An Introduction for Applied Mathematicianscited by 5
  28. 2024Stable Learning Using Spiking Neural Networks Equipped With Affine Encoders and Decoderscited by 5
  29. 2026TASR: Training-Free Adaptive Stopping for Iterative Retrievalcited by 5
  30. 2024Optical Diffusion Models for Image Generationcited by 4
  31. 2024AI, Climate, and Transparency: Operationalizing and Improving the AI Actcited by 4
  32. 2024AI, Climate, and Regulation: From Data Centers to the AI Actcited by 4
  33. 2024Dynamic layer selection in decoder-only transformerscited by 4
  34. 2025Compressing Language Models for Specialized Domainscited by 4
  35. 2025Reusing Computation in Text-to-Image Diffusion for Efficient Generation of Image Setscited by 4
  36. 2025Who Evaluates AI's Social Impacts? Mapping Coverage and Gaps in First and Third Party Evaluationscited by 4
  37. 2025Context-aware LLM-based AI Agents for Human-centered Energy Management Systems in Smart Buildingscited by 4
  38. 2026Understanding Efficiency: Quantization, Batching, and Serving Strategies in LLM Energy Usecited by 4

50 most cited of 191 All 191 (JSON)