mlsanityData to 14 Oct 2026

From Prompts to Power: Measuring the Energy Footprint of LLM Inference

arXiv 2511.05597 ↗First posted 5 Nov 2025cs.AI cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

The rapid expansion of Large Language Models (LLMs) has introduced unprecedented energy demands, extending beyond training to large-scale inference workloads that often dominate total lifecycle consumption. Deploying these models requires energy-intensive GPU infrastructure, and in some cases has even prompted plans to power data centers with nuclear energy. Despite this growing relevance, systematic analyses of inference energy consumption remain limited. In this work, we present a large-scale measurement-based study comprising over 32,500 measurements across 21 GPU configurations and 155 model architectures, from small open-source models to frontier systems. Using the vLLM inference engine, we quantify energy usage at the prompt level and identify how architectural and operational factors shape energy demand. Building on these insights, we develop a predictive model that accurately estimates inference energy consumption across unseen architectures and hardware, and implement it as a browser extension to raise awareness of the environmental impact of generative AI.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 44 of its 71 references that are in the map. First posted on arXiv 5 Nov 2025.

Builds on

44 of 71 references.

References · 44

  1. 2017Attention Is All You Needcited by 60,361
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  6. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  7. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  8. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  9. 2023Qwen Technical Reportcited by 3,344
  10. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  11. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  12. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
32 more
  1. 2024Mixtral of Expertscited by 1,823
  2. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  3. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  4. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,184
  5. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  6. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
  7. 2024Gemma: Open Models Based on Gemini Research and Technologycited by 916
  8. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 717
  9. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 713
  10. 2024Phi-4 Technical Reportcited by 711
  11. 2023A Comprehensive Overview of Large Language Modelscited by 648
  12. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 313
  13. 2022Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Modelcited by 265
  14. 2021Sustainable AI: Environmental Implications, Challenges and Opportunitiescited by 242
  15. 2023From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inferencecited by 170
  16. 2020Carbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Modelscited by 169
  17. 2022Zeus: Understanding and Optimizing GPU Energy Consumption of DNN Trainingcited by 79
  18. 2023LLMCarbon: Modeling the end-to-end Carbon Footprint of Large Language Modelscited by 72
  19. 2023EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelismcited by 69
  20. 2024Towards Greener LLMs: Bringing Energy-Efficiency to the Forefront of LLM Inferencecited by 61
  21. 2022Eco2AI: carbon emissions tracking of machine learning models as the first step towards sustainable AIcited by 58
  22. 2024A Performance Study of LLM-Generated Code on Leetcodecited by 49
  23. 2024Pushing The Limit of LLM Capacity for Text Classificationcited by 28
  24. 2024Offline Energy-Optimal LLM Serving: Workload-Based Energy Models for LLM Inference on Heterogeneous Systemscited by 28
  25. 2024The Vulnerability of Language Model Benchmarks: Do They Accurately Reflect True LLM Performance?cited by 22
  26. 2023Benchmarking LLM powered Chatbots: Methods and Metricscited by 19
  27. 2024LLMCO2: Advancing Accurate Carbon Footprint Prediction for LLM Inferencescited by 19
  28. 2024Towards Sustainable Large Language Model Servingcited by 12
  29. 2025AdaSkip: Adaptive Sublayer Skipping for Accelerating Long-Context LLM Inferencecited by 12
  30. 2024AI Benchmarks and Datasets for LLM Evaluationcited by 7
  31. 2024Performance Law of Large Language Modelscited by 6
  32. 2025GPTFootprint: Increasing Consumer Awareness of the Environmental Impacts of LLMscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 12.

12 papers.

  1. 2026From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUscited by 3
  2. 2026Pimp My LLM: Leveraging Variability Modeling to Tune Inference Hyperparameterscited by 1
  3. 2026WattGPU: Predicting Inference Power and Latency on Unseen GPUs and LLMscited by 1
  4. 2026Request-Level Energy Attribution for Batched LLM Servingcited by 1
  5. 2026COMPASS: The explainable agentic framework for Sovereignty, Sustainability, Compliance, and Ethicscited by 0
  6. 2026EnergyLens: Interpretable Closed-Form Energy Models for Multimodal LLM Inference Servingcited by 0
  7. 2026Algorithmic algorithm development with LLMs: A Case Study on LLM-Usage for Contraction Order Optimization in Tensor Networkscited by 0
  8. 2026ASTRA-sim 3.0: Next-Level Distributed Machine Learning Simulations via High-Fidelity GPU and Infrastructure Modelingcited by 0
  9. 2026A Picture is Worth a Thousand Tokens: How Vision Language Models Cut AI Energy Costs While Improving Accuracycited by 0
  10. 2026From Caveman to Expert Analyst: Energy Consumption of Variable LLM Taskscited by 0
  11. 2026TokenPowerSandbox: Evidence-Gated CPU-First Screening for Energy-Aware LLM Servingcited by 0
  12. 2026How Do Prompt Variations Affect Energy Consumption in On-Device LLMs?cited by 0