mlsanityData to 7 Oct 2026

Towards Sustainable Large Language Model Serving

arXiv 2501.01990 ↗First posted 31 Dec 2024cs.LG cs.DCcited by 12

Not on the map. Why?

Abstract (arXiv)

In this work, we study LLMs from a carbon emission perspective, addressing both operational and embodied emissions, and paving the way for sustainable LLM serving. We characterize the performance and energy of LLaMA with 1B, 3B, and 7B parameters using two Nvidia GPU types, a latest-generation RTX6000 Ada and an older-generation T4. We analytically model operational carbon emissions based on energy consumption and carbon intensities from three grid regions -- each representing a different energy source mix, and embodied carbon emissions based on chip area and memory size. Our characterization and modeling provide us with an in-depth understanding of the performance, energy, and carbon emissions of LLM serving. Our findings highlight the potential for optimizing sustainable LLM serving systems by considering both operational and embodied carbon emissions simultaneously.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 15 of its 36 references that are in the map. First posted on arXiv 31 Dec 2024.

Builds on

15 of 36 references.

References · 15

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  3. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  4. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  5. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 595
  6. 2022Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Modelcited by 264
  7. 2024RAFT: Adapting Language Model to Domain Specific RAGcited by 258
  8. 2023PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPUcited by 210
  9. 2022The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrinkcited by 183
  10. 2023LLMCarbon: Modeling the end-to-end Carbon Footprint of Large Language Modelscited by 71
  11. 2023Counting Carbon: A Survey of Factors Influencing the Emissions of Machine Learningcited by 42
  12. 2023Small Models are Valuable Plug-ins for Large Language Modelscited by 42
3 more
  1. 2023Small Language Models Fine-tuned to Coordinate Larger Language Models improve Complex Reasoningcited by 16
  2. 2023Toward Sustainable HPC: Carbon Footprint Estimation and Environmental Implications of HPC Systemscited by 12
  3. 2023Embodied Carbon Accounting through Spatial-Temporal Embodied Carbon Modelscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 6.

12 papers.

  1. 2025Taming the Titans: A Survey of Efficient LLM Inference Servingcited by 16
  2. 2025From Prompts to Power: Measuring the Energy Footprint of LLM Inferencecited by 12
  3. 2025Unveiling Environmental Impacts of Large Language Model Serving: A Functional Unit Viewcited by 11
  4. 2025Systematic Characterization of LLM Quantization: A Performance, Energy, and Quality Perspectivecited by 8
  5. 2025EnsembleCI: Ensemble Learning for Carbon Intensity Forecastingcited by 5
  6. 2026The Energy Footprint of LLM-Based Environmental Analysis: LLMs and Domain Productscited by 2
  7. 2025The Hidden AI Race: Tracking Environmental Costs of Innovationcited by 1
  8. 2026BIRDS: Characterizing and Understanding Biodiversity Impact of Large Language Model Servingcited by 1
  9. 2026Evaluation of ML Resource Utilization Requires Model Life Cycle Assessmentcited by 0
  10. 2026From Perception to Action: Can UI Interventions Foster Sustainable LLM Chatbotcited by 0
  11. 2026AgentDecarbonizer: Carbon-Aware Execution for AI Agentscited by 0
  12. 2026Beyond Energy: When Sustainability Dimensions Reshape LLM Serving Decisionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024GreenLLM: Disaggregating Large Language Model Serving on Heterogeneous GPUs for Lower Carbon Emissions
  2. 2026Watt Counts: Energy-Aware Benchmark for Sustainable LLM Inference on Heterogeneous GPU Architectures
  3. 2024Harnessing Your DRAM and SSD for Sustainable and Accessible LLM Inference with Mixed-Precision and Multi-level Caching
  4. 2025Cache Your Prompt When It's Green: Carbon-Aware Caching for Large Language Model Serving
  5. 2025CarbonCall: Sustainability-Aware Function Calling for Large Language Models on Edge Devices
  6. 2024Towards Greener LLMs: Bringing Energy-Efficiency to the Forefront of LLM Inference
  7. 2025EcoServe: Designing Carbon-Aware AI Inference Systems
  8. 2024Large Language Models (LLMs): Deployment, Tokenomics and Sustainability