mlsanityData to 14 Oct 2026

Dissecting the Carbon-SLO Tension in Geo-Distributed LLM Serving

arXiv 2610.14571 ↗First posted 10 Oct 2026cs.DC cs.LG cs.PFcited by 0

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

Large Language Model (LLM) inference is driving a growing share of datacenter carbon emissions. Although carbon-aware scheduling can reduce these emissions in geo-distributed datacenters by exploiting the spatiotemporal variations in grid carbon intensity, the latency-sensitive nature of LLM serving constrains this design space. While prior studies investigate the carbon savings achievable through spatial scheduling, they oversimplify the dynamics of modern inference engines, such as queueing and continuous batching, which strongly shape inference latency. We address this issue by designing and evaluating a latency-centric, geo-distributed inference simulator, calibrated with hardware-grounded latency and power measurements, that reproduces these dynamics and enables the study of carbon savings under strict service-level objectives (SLOs). We then formulate a series of SLO- and carbon-aware schedulers of increasing sophistication, from per-request heuristics to batch-level optimization: (i) a greedy heuristic, (ii) an integer linear programming (ILP)-based scheduler jointly assigning batches of requests, (iii) an iterative ILP-based solution projecting its own assignments back into the load estimates to capture load coupling, and (iv) a facility mixed-integer linear programming (MILP) scheduler pricing the energy of batched requests collectively rather than in isolation. Across three LLMs under MLPerf SLOs and varying system loads, the MILP-based option offers the best carbon-SLO balance, reducing carbon emissions by up to 60% relative to the round-robin policy, with an average SLO violation of 3%. Our characterization further shows that savings grow with the spread of carbon intensity across datacenters and shrink as system load increases, while SLO attainment depends on how frequently schedulers receive datacenters' load updates.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 49 of its 89 references that are in the map. First posted on arXiv 10 Oct 2026.

Builds on

49 of 89 references.

Closest ancestors · 8

Same direction, newest first

  1. 2026LLMServingSim 2.0: A Unified Simulator for Heterogeneous and Disaggregated LLM Serving Infrastructure26 Feb
  2. 2026DualScale: Energy-Efficient Disaggregated LLM Serving via Phase-Aware Placement and DVFS21 Feb
  3. 2024LLMServingSim: A HW/SW Co-Simulation Infrastructure for LLM Inference Serving at Scale10 Aug
  4. 2024SLO-aware GPU Frequency Scaling for Energy Efficient LLM Inference Serving5 Aug
  5. 2024Llumnix: Dynamic Scheduling for Large Language Model Serving5 Jun
  6. 2024Efficient Interactive LLM Serving with Proxy Model-based Sequence Length Prediction12 Apr
  7. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve4 Mar
  8. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving18 Jan

Other references · 41

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  7. 2024Qwen2.5 Technical Reportcited by 4,942
  8. 2021Program Synthesis with Large Language Modelscited by 3,637
  9. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
  10. 2024Mixtral of Expertscited by 1,823
  11. 2015Teaching Machines to Read and Comprehendcited by 1,049
  12. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 558
29 more
  1. 2021Efficient Attentions for Long Document Summarizationcited by 345
  2. 2019MLPerf Inference Benchmarkcited by 199
  3. 2023Power Hungry Processing: Watts Driving the Cost of AI Deployment?cited by 191
  4. 2024Vidur: A Large-Scale Simulation Framework For LLM Inferencecited by 117
  5. 2024DynamoLLM: Designing LLM Inference Clusters for Performance and Energy Efficiencycited by 112
  6. 2015Parallelizing the dual revised simplex methodcited by 89
  7. 2020Chasing Carbon: The Elusive Environmental Footprint of Computingcited by 72
  8. 2024LLMs as Research Tools: A Large Scale Survey of Researchers' Usage and Perceptionscited by 47
  9. 2021Carbon-Aware Computing for Datacenterscited by 41
  10. 2022Carbon Explorer: A Holistic Approach for Designing Carbon Aware Datacenterscited by 32
  11. 2025The European Union general data protection regulation: what it is and what it meanscited by 31
  12. 2025EcoServe: Designing Carbon-Aware AI Inference Systemscited by 19
  13. 2025Working with AI: Measuring the Applicability of Generative AI to Occupationscited by 19
  14. 2023CarbonScaler: Leveraging Cloud Workload Elasticity for Optimizing Carbon-Efficiencycited by 17
  15. 2023Clover: Toward Sustainable AI with Carbon-Aware Machine Learning Inference Servicecited by 17
  16. 2021Let's Wait Awhile: How Temporal Workload Shifting Can Reduce Carbon Emissions in the Cloudcited by 14
  17. 2025SkyWalker: A Locality-Aware Cross-Region Load Balancer for LLM Inferencecited by 10
  18. 2023On the Limitations of Carbon-Aware Temporal and Spatial Workload Shifting in the Cloudcited by 8
  19. 2024CASPER: Carbon-Aware Scheduling and Provisioning for Distributed Web Servicescited by 8
  20. 2022Ecovisor: A Virtual Energy System for Carbon-Efficient Applicationscited by 7
  21. 2020Energy Efficient Computing Systems: Architectures, Abstractions and Modeling to Techniques and Standardscited by 6
  22. 2025Sustainable Carbon-Aware and Water-Efficient LLM Scheduling in Geo-Distributed Cloud Datacenterscited by 4
  23. 2025FREESH: Fair, Resource- and Energy-Efficient Scheduling for LLM Serving on Heterogeneous GPUscited by 3
  24. 2025Green-LLM: Optimal Workload Allocation for Environmentally-Aware Distributed Inferencecited by 2
  25. 2026Routing LLM Inference to the Cleanest Grid in Real Timecited by 2
  26. 2024Learning-Augmented Competitive Algorithms for Spatiotemporal Online Allocation with Deadline Constraintscited by 1
  27. 2026Parallelization Strategies for Dense LLM Deployment: Navigating Through Application-Specific Tradeoffs and Bottleneckscited by 1
  28. 2026GAR: Carbon-Aware Routing for LLM Inference via Constrained Optimizationcited by 1
  29. 2026MARLIN: Multi-Agent Game-Theoretic Reinforcement Learning for Sustainable LLM Inference in Cloud Datacenterscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Towards Greener LLMs: Bringing Energy-Efficiency to the Forefront of LLM Inference
  2. 2026Cascade: Exploiting SLO-Aware latency budget for fair and high goodput LLM inference serving
  3. 2025Cache Your Prompt When It's Green: Carbon-Aware Caching for Large Language Model Serving
  4. 2026Carbon-Aware Routing for Function Calling in Edge-Cloud LLM Systems
  5. 2026Position: LLM Serving Needs Mathematical Optimization and Algorithmic Foundations, Not Just Heuristics
  6. 2026Understanding Efficiency: Quantization, Batching, and Serving Strategies in LLM Energy Use
  7. 2025TAPAS: Thermal- and Power-Aware Scheduling for LLM Inference in Cloud Platforms
  8. 2026From Inference Engine to Inference Control Plane: Connecting vLLM, llm-d, and the Evolution of Efficient Distributed LLM Serving