# Towards Sustainable Large Language Model Serving

arXiv: https://arxiv.org/abs/2501.01990 · HTML: https://mlsanity.com/paper/2501.01990/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 36 references that are in the map.
- First posted on arXiv 31 Dec 2024.

## Builds on (most cited first, 15 of 15)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Fast Inference from Transformers via Speculative Decoding](https://mlsanity.com/paper/2211.17192/) (2022)
- [Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation](https://mlsanity.com/paper/2305.01210/) (2023)
- [HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face](https://mlsanity.com/paper/2303.17580/) (2023)
- [DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving](https://mlsanity.com/paper/2401.09670/) (2024)
- [Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model](https://mlsanity.com/paper/2211.02001/) (2022)
- [RAFT: Adapting Language Model to Domain Specific RAG](https://mlsanity.com/paper/2403.10131/) (2024)
- [PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU](https://mlsanity.com/paper/2312.12456/) (2023)
- [The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink](https://mlsanity.com/paper/2204.05149/) (2022)
- [LLMCarbon: Modeling the end-to-end Carbon Footprint of Large Language Models](https://mlsanity.com/paper/2309.14393/) (2023)
- [Counting Carbon: A Survey of Factors Influencing the Emissions of Machine Learning](https://mlsanity.com/paper/2302.08476/) (2023)
- [Small Models are Valuable Plug-ins for Large Language Models](https://mlsanity.com/paper/2305.08848/) (2023)
- [Small Language Models Fine-tuned to Coordinate Larger Language Models improve Complex Reasoning](https://mlsanity.com/paper/2310.18338/) (2023)
- [Toward Sustainable HPC: Carbon Footprint Estimation and Environmental Implications of HPC Systems](https://mlsanity.com/paper/2306.13177/) (2023)
- [Embodied Carbon Accounting through Spatial-Temporal Embodied Carbon Models](https://mlsanity.com/paper/2312.06364/) (2023)

## Built on by (most cited first, 12 of 12)

- [Taming the Titans: A Survey of Efficient LLM Inference Serving](https://mlsanity.com/paper/2504.19720/) (2025)
- [From Prompts to Power: Measuring the Energy Footprint of LLM Inference](https://mlsanity.com/paper/2511.05597/) (2025)
- [Unveiling Environmental Impacts of Large Language Model Serving: A Functional Unit View](https://mlsanity.com/paper/2502.11256/) (2025)
- [Systematic Characterization of LLM Quantization: A Performance, Energy, and Quality Perspective](https://mlsanity.com/paper/2508.16712/) (2025)
- [EnsembleCI: Ensemble Learning for Carbon Intensity Forecasting](https://mlsanity.com/paper/2505.01959/) (2025)
- [The Energy Footprint of LLM-Based Environmental Analysis: LLMs and Domain Products](https://mlsanity.com/paper/2604.00053/) (2026)
- [The Hidden AI Race: Tracking Environmental Costs of Innovation](https://mlsanity.com/paper/2511.22781/) (2025)
- [BIRDS: Characterizing and Understanding Biodiversity Impact of Large Language Model Serving](https://mlsanity.com/paper/2605.27480/) (2026)
- [Evaluation of ML Resource Utilization Requires Model Life Cycle Assessment](https://mlsanity.com/paper/2606.07632/) (2026)
- [From Perception to Action: Can UI Interventions Foster Sustainable LLM Chatbot](https://mlsanity.com/paper/2606.10861/) (2026)
- [AgentDecarbonizer: Carbon-Aware Execution for AI Agents](https://mlsanity.com/paper/2608.20566/) (2026)
- [Beyond Energy: When Sustainability Dimensions Reshape LLM Serving Decisions](https://mlsanity.com/paper/2609.35569/) (2026)

## Abstract (arXiv)

In this work, we study LLMs from a carbon emission perspective, addressing both operational and embodied emissions, and paving the way for sustainable LLM serving. We characterize the performance and energy of LLaMA with 1B, 3B, and 7B parameters using two Nvidia GPU types, a latest-generation RTX6000 Ada and an older-generation T4. We analytically model operational carbon emissions based on energy consumption and carbon intensities from three grid regions -- each representing a different energy source mix, and embodied carbon emissions based on chip area and memory size. Our characterization and modeling provide us with an in-depth understanding of the performance, energy, and carbon emissions of LLM serving. Our findings highlight the potential for optimizing sustainable LLM serving systems by considering both operational and embodied carbon emissions simultaneously.