# LLMCO2: Advancing Accurate Carbon Footprint Prediction for LLM Inferences

arXiv: https://arxiv.org/abs/2410.02950 · HTML: https://mlsanity.com/paper/2410.02950/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 35 references that are in the map.
- First posted on arXiv 3 Oct 2024.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints](https://mlsanity.com/paper/2305.13245/) (2023)
- [Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM](https://mlsanity.com/paper/2104.04473/) (2021)
- [Splitwise: Efficient generative LLM inference using phase splitting](https://mlsanity.com/paper/2311.18677/) (2023)
- [Efficiently Scaling Transformer Inference](https://mlsanity.com/paper/2211.05102/) (2022)
- [Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve](https://mlsanity.com/paper/2403.02310/) (2024)
- [DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale](https://mlsanity.com/paper/2207.00032/) (2022)
- [Using an LLM to Help With Code Understanding](https://mlsanity.com/paper/2307.08177/) (2023)
- [Power Hungry Processing: Watts Driving the Cost of AI Deployment?](https://mlsanity.com/paper/2311.16863/) (2023)
- [LLM Inference Unveiled: Survey and Roofline Model Insights](https://mlsanity.com/paper/2402.16363/) (2024)
- [On Extractive and Abstractive Neural Document Summarization with Transformer Language Models](https://mlsanity.com/paper/1909.03186/) (2019)
- [LLMCarbon: Modeling the end-to-end Carbon Footprint of Large Language Models](https://mlsanity.com/paper/2309.14393/) (2023)
- [Unveiling Energy Efficiency in Deep Learning: Measurement, Prediction, and Scoring across Edge Devices](https://mlsanity.com/paper/2310.18329/) (2023)

## Built on by (most cited first, 19 of 19)

- [A Survey on Inference Optimization Techniques for Mixture of Experts Models](https://mlsanity.com/paper/2412.14219/) (2024)
- [A Survey of Sustainability in Large Language Models: Applications, Economics, and Challenges](https://mlsanity.com/paper/2412.04782/) (2024)
- [From Prompts to Power: Measuring the Energy Footprint of LLM Inference](https://mlsanity.com/paper/2511.05597/) (2025)
- [Unveiling Environmental Impacts of Large Language Model Serving: A Functional Unit View](https://mlsanity.com/paper/2502.11256/) (2025)
- [Watt Counts: Energy-Aware Benchmark for Sustainable LLM Inference on Heterogeneous GPU Architectures](https://mlsanity.com/paper/2604.09048/) (2026)
- [From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUs](https://mlsanity.com/paper/2607.26571/) (2026)
- [The Energy Footprint of LLM-Based Environmental Analysis: LLMs and Domain Products](https://mlsanity.com/paper/2604.00053/) (2026)
- [Pimp My LLM: Leveraging Variability Modeling to Tune Inference Hyperparameters](https://mlsanity.com/paper/2602.17697/) (2026)
- [Multilinguality at the Edge: Developing Language Models for the Global South](https://mlsanity.com/paper/2604.21637/) (2026)
- [WattGPU: Predicting Inference Power and Latency on Unseen GPUs and LLMs](https://mlsanity.com/paper/2607.02391/) (2026)
- [Attention to Detail: Evaluating Energy, Performance, and Accuracy Trade-offs Across vLLM Configurations](https://mlsanity.com/paper/2607.09172/) (2026)
- [Request-Level Energy Attribution for Batched LLM Serving](https://mlsanity.com/paper/2608.00026/) (2026)
- [AOLO: Analysis and Optimization For Low-Carbon Oriented Wireless Large Language Model Services](https://mlsanity.com/paper/2503.04418/) (2025)
- [SEALing the Gap: A Reference Framework for LLM Inference Carbon Estimation via Multi-Benchmark Driven Embodiment](https://mlsanity.com/paper/2603.02949/) (2026)
- [LLMSpace: Carbon Footprint Modeling for Large Language Model Inference on LEO Satellites](https://mlsanity.com/paper/2605.05615/) (2026)
- [EnergyLens: Interpretable Closed-Form Energy Models for Multimodal LLM Inference Serving](https://mlsanity.com/paper/2605.10556/) (2026)
- [EnergyLens: Predictive Energy-Aware Exploration for Multi-GPU LLM Inference Optimization](https://mlsanity.com/paper/2605.14249/) (2026)
- [Quantifying the Relationship Between Clinical Safety and Environmental Impact in Therapeutic LLMs](https://mlsanity.com/paper/2608.11830/) (2026)
- [TokenPowerSandbox: Evidence-Gated CPU-First Screening for Energy-Aware LLM Serving](https://mlsanity.com/paper/2608.18149/) (2026)

## Abstract (arXiv)

Throughout its lifecycle, a large language model (LLM) generates a substantially larger carbon footprint during inference than training. LLM inference requests vary in batch size, prompt length, and token generation number, while cloud providers employ different GPU types and quantities to meet diverse service-level objectives for accuracy and latency. It is crucial for both users and cloud providers to have a tool that quickly and accurately estimates the carbon impact of LLM inferences based on a combination of inference request and hardware configurations before execution. Estimating the carbon footprint of LLM inferences is more complex than training due to lower and highly variable model FLOPS utilization, rendering previous equation-based models inaccurate. Additionally, existing machine learning (ML) prediction methods either lack accuracy or demand extensive training data, as they inadequately handle the distinct prefill and decode phases, overlook hardware-specific features, and inefficiently sample uncommon inference configurations. We introduce coo, a graph neural network (GNN)-based model that greatly improves the accuracy of LLM inference carbon footprint predictions compared to previous methods.