mlsanityData to 14 Oct 2026

LLMCO2: Advancing Accurate Carbon Footprint Prediction for LLM Inferences

arXiv 2410.02950 ↗First posted 3 Oct 2024cs.LG cs.AI cs.CLcited by 19

Not on the map. Why?

Abstract (arXiv)

Throughout its lifecycle, a large language model (LLM) generates a substantially larger carbon footprint during inference than training. LLM inference requests vary in batch size, prompt length, and token generation number, while cloud providers employ different GPU types and quantities to meet diverse service-level objectives for accuracy and latency. It is crucial for both users and cloud providers to have a tool that quickly and accurately estimates the carbon impact of LLM inferences based on a combination of inference request and hardware configurations before execution. Estimating the carbon footprint of LLM inferences is more complex than training due to lower and highly variable model FLOPS utilization, rendering previous equation-based models inaccurate. Additionally, existing machine learning (ML) prediction methods either lack accuracy or demand extensive training data, as they inadequately handle the distinct prefill and decode phases, overlook hardware-specific features, and inefficiently sample uncommon inference configurations. We introduce coo, a graph neural network (GNN)-based model that greatly improves the accuracy of LLM inference carbon footprint predictions compared to previous methods.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 18 of its 35 references that are in the map. First posted on arXiv 3 Oct 2024.

Builds on

18 of 35 references.

References · 18

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2017Inductive Representation Learning on Large Graphscited by 6,320
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  5. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  6. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  7. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  8. 2021Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LMcited by 632
  9. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 558
  10. 2022Efficiently Scaling Transformer Inferencecited by 539
  11. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Servecited by 526
  12. 2022DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scalecited by 407
6 more
  1. 2023Using an LLM to Help With Code Understandingcited by 228
  2. 2023Power Hungry Processing: Watts Driving the Cost of AI Deployment?cited by 191
  3. 2024LLM Inference Unveiled: Survey and Roofline Model Insightscited by 152
  4. 2019On Extractive and Abstractive Neural Document Summarization with Transformer Language Modelscited by 83
  5. 2023LLMCarbon: Modeling the end-to-end Carbon Footprint of Large Language Modelscited by 72
  6. 2023Unveiling Energy Efficiency in Deep Learning: Measurement, Prediction, and Scoring across Edge Devicescited by 17

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 3; 2026: 14.

19 papers.

Sort:

By area:

  1. 2024A Survey on Inference Optimization Techniques for Mixture of Experts Modelscited by 47
  2. 2024A Survey of Sustainability in Large Language Models: Applications, Economics, and Challengescited by 18
  3. 2025From Prompts to Power: Measuring the Energy Footprint of LLM Inferencecited by 12
  4. 2025Unveiling Environmental Impacts of Large Language Model Serving: A Functional Unit Viewcited by 11
  5. 2026Watt Counts: Energy-Aware Benchmark for Sustainable LLM Inference on Heterogeneous GPU Architecturescited by 5
  6. 2026From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUscited by 3
  7. 2026The Energy Footprint of LLM-Based Environmental Analysis: LLMs and Domain Productscited by 2
  8. 2026Pimp My LLM: Leveraging Variability Modeling to Tune Inference Hyperparameterscited by 1
  9. 2026Multilinguality at the Edge: Developing Language Models for the Global Southcited by 1
  10. 2026WattGPU: Predicting Inference Power and Latency on Unseen GPUs and LLMscited by 1
  11. 2026Attention to Detail: Evaluating Energy, Performance, and Accuracy Trade-offs Across vLLM Configurationscited by 1
  12. 2026Request-Level Energy Attribution for Batched LLM Servingcited by 1
7 more
  1. 2025AOLO: Analysis and Optimization For Low-Carbon Oriented Wireless Large Language Model Servicescited by 0
  2. 2026SEALing the Gap: A Reference Framework for LLM Inference Carbon Estimation via Multi-Benchmark Driven Embodimentcited by 0
  3. 2026LLMSpace: Carbon Footprint Modeling for Large Language Model Inference on LEO Satellitescited by 0
  4. 2026EnergyLens: Interpretable Closed-Form Energy Models for Multimodal LLM Inference Servingcited by 0
  5. 2026EnergyLens: Predictive Energy-Aware Exploration for Multi-GPU LLM Inference Optimizationcited by 0
  6. 2026Quantifying the Relationship Between Clinical Safety and Environmental Impact in Therapeutic LLMscited by 0
  7. 2026TokenPowerSandbox: Evidence-Gated CPU-First Screening for Energy-Aware LLM Servingcited by 0