mlsanityData to 7 Oct 2026

VoltanaLLM: Energy-Efficient and SLO-Aware Disaggregated LLM Serving via Adaptive Frequency Control and State-Space Routing

arXiv 2509.04827 ↗First posted 5 Sep 2025cs.DC cs.AI cs.LGcited by 2

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

The energy cost of Large Language Model (LLM) inference is rapidly becoming a barrier to sustainable and scalable deployment. Although modern serving architectures expose distinct prefill and decode behaviors, existing systems fail to exploit these phase differences for energy-efficient serving under strict latency SLOs. This paper introduces VoltanaLLM, the first system that explicitly targets and reduces the energy bloat in modern prefill-decode (P/D) disaggregated LLM serving. Guided by a control-theory perspective, VoltanaLLM separates two levers: per-instance operating-point selection (GPU frequency per iteration) and system-level state-space routing of requests. We empirically observe that LLM inference exhibits a U-shaped energy-frequency curve creating "sweet spots" that depend on phase behavior and load. VoltanaLLM exploits this by combining phase-specific, iteration-level frequency selection driven by a lightweight, online-adaptive latency predictor, with a decode state-space guided router that avoids architectural granularity-induced inefficiencies, all while meeting desired SLOs. We implement VoltanaLLM using SGLang and evaluate it across multiple models and real-world workloads. Our results show VoltanaLLM reduces end-to-end energy by up to 36.3% versus a static max-frequency baseline while maintaining high SLO attainment, and generalizes to newer GPUs. These results point to sustainable LLM serving via phase-aware, iteration-level frequency selection coupled with architecture-aware routing. Source code is available in https://github.com/Supercomputing-System-AI-Lab/VoltanaLLM.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 35 of its 57 references that are in the map. First posted on arXiv 5 Sep 2025.

Builds on

35 of 57 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024SLO-aware GPU Frequency Scaling for Energy Efficient LLM Inference Serving5 Aug
  2. 2024Llumnix: Dynamic Scheduling for Large Language Model Serving5 Jun
  3. 2024vAttention: Dynamic Memory Management for Serving LLMs without PagedAttention7 May
  4. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve4 Mar
  5. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving18 Jan
  6. 2023Fast Distributed Inference Serving for Large Language Models10 May

Other references · 29

  1. 2017Attention Is All You Needcited by 59,932
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  4. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  5. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  6. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  7. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  8. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  9. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 567
  10. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 536
  11. 2022Efficiently Scaling Transformer Inferencecited by 529
  12. 2024FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisioncited by 507
17 more
  1. 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 449
  2. 2022DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scalecited by 403
  3. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verificationcited by 359
  4. 2018Blockwise Parallel Decoding for Deep Autoregressive Modelscited by 318
  5. 2023S-LoRA: Serving Thousands of Concurrent LoRA Adapterscited by 146
  6. 2018Learning to Optimize Tensor Programscited by 138
  7. 2024Inference without Interference: Disaggregate LLM Inference for Mixed Downstream Workloadscited by 120
  8. 2024WebPilot: A Versatile and Autonomous Multi-Agent System for Web Task Execution with Strategic Explorationcited by 100
  9. 2020TurboTransformers: An Efficient GPU Serving System For Transformer Modelscited by 71
  10. 2024Towards Greener LLMs: Bringing Energy-Efficiency to the Forefront of LLM Inferencecited by 59
  11. 2025Energy Considerations of Large Language Model Inference and Efficiency Optimizationscited by 54
  12. 2023AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Servingcited by 31
  13. 2025Measuring the environmental impact of delivering AI at Google Scalecited by 31
  14. 2025TAPAS: Thermal- and Power-Aware Scheduling for LLM Inference in Cloud Platformscited by 28
  15. 2025EcoServe: Designing Carbon-Aware AI Inference Systemscited by 18
  16. 2024Pie: Pooling CPU Memory for LLM Inferencecited by 13
  17. 2025AI Greenferencing: Routing AI Inferencing to Green Modular Data Centers with Heroncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Phase-Decoupled, Model-Calibrated Power Control for Disaggregated LLM Servingcited by 0
  2. 2026Analytical Power-Aware Provisioning for Prefill-Decode Disaggregated AI Inferencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026SLO-Aware Compute Resource Allocation for Prefill-Decode Disaggregated LLM Inference
  2. 2025Arrow: Adaptive Scheduling Mechanisms for Disaggregated LLM Inference Architecture
  3. 2025DOPD: A Dynamic PD-Disaggregation Architecture for Maximizing Goodput in LLM Inference Serving
  4. 2026Cascade: Exploiting SLO-Aware latency budget for fair and high goodput LLM inference serving
  5. 2026Energy-Efficient LLM Serving via Disaggregated Attention--FFN and Flexible Frequency Scaling
  6. 2024ScaleLLM: A Resource-Frugal LLM Serving Framework by Optimizing End-to-End Efficiency
  7. 2025Harli: SLO-Aware Co-location of LLM Inference and PEFT-based Finetuning on Model-as-a-Service Platforms
  8. 2025Optimal Scheduling Algorithms for LLM Inference: Theory and Practice