Analytical Power-Aware Provisioning for Prefill-Decode Disaggregated AI Inference
On the map: Efficient LLM inference and serving → Serving systems and request scheduling
Abstract (arXiv)
Power availability increasingly constrains the operation of AI inference fleets, creating a need for provisioning methods that jointly consider serving capacity and power consumption. Prefill--decode (PD) disaggregation has emerged as a prevalent architecture for large-scale inference serving. However, determining the appropriate numbers of prefill and decode instances is challenging because serving capacity depends jointly on workload characteristics, hardware constraints, queueing, and KV-cache reservations. Existing approaches largely rely on profiling and simulation, providing limited analytical insight into how provisioning decisions shape the tradeoff between serving capacity and power consumption. This paper develops an analytical framework for power-aware provisioning of PD-disaggregated AI inference. Given an inference workload and hardware, the framework models the serving capacity and average power consumption of a provisioned deployment. The serving-capacity model is derived from the joint distribution of input--output lengths and hardware compute and memory limits. In particular, it explicitly captures the coupling between prefill and decode induced by KV-cache reservations, as well as the impact of request queueing. On this basis, the power model determines per-instance power consumption as a function of normalized serving throughput. Together, the models determine the serving capacity--power Pareto front among candidate provisioned deployments, enabling the service provider to choose a provisioned deployment as the workload or available power changes.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 30 of its 57 references that are in the map. First posted on arXiv 21 Sep 2026.
Builds on
30 of 57 references.
Closest ancestors · 12
Same direction, newest first
- 2026PowerSlider: Exploiting Phase Asymmetry for LLM Serving under Demand Response22 Aug
- 2026Frontier: Towards Comprehensive and Accurate LLM Inference Simulation20 May
- 2026A Queueing-Theoretic Framework for Stability Analysis of LLM Inference with KV Cache Memory Constraints6 May
- 2026Watt Counts: Energy-Aware Benchmark for Sustainable LLM Inference on Heterogeneous GPU Architectures10 Apr
- 2026FleetOpt: Analytical Fleet Provisioning for LLM Inference with Compress-and-Route as Implementation Mechanism17 Mar
- 2026SLO-Aware Compute Resource Allocation for Prefill-Decode Disaggregated LLM Inference5 Mar
- 2026DualScale: Energy-Efficient Disaggregated LLM Serving via Phase-Aware Placement and DVFS21 Feb
- 2025TokenScale: Timely and Accurate Autoscaling for Disaggregated LLM Serving with Token Velocity3 Dec
- 2025VoltanaLLM: Energy-Efficient and SLO-Aware Disaggregated LLM Serving via Adaptive Frequency Control and State-Space Routing5 Sep
- 2025Arrow: Adaptive Scheduling Mechanisms for Disaggregated LLM Inference Architecture17 May
- 2024Offline Energy-Optimal LLM Serving: Workload-Based Energy Models for LLM Inference on Heterogeneous Systems4 Jul
- 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve4 Mar
Other references · 18
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
- 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
- 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 536
- 2022The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrinkcited by 183
- 2024Vidur: A Large-Scale Simulation Framework For LLM Inferencecited by 115
- 2025TokenPowerBench: Benchmarking the Power Consumption of LLM Inferencecited by 21
- 2025Optimizing LLM Inference: Fluid-Guided Online Scheduling with Memory Constraintscited by 16
- 2026Agentic Coding in the Wild: Characterizing GitHub Copilot Traces at Production Scalecited by 9
- 2026SweetSpot: An Analytical Model for Predicting Energy Efficiency of LLM Inferencecited by 6
- 2025Taming the Chaos: Coordinated Autoscaling for Heterogeneous and Disaggregated LLM Inferencecited by 5
- 2025BestServe: Serving Strategies with Optimal Goodput in Collocation and Disaggregation Architecturescited by 3
6 more
- 2026From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUscited by 3
- 2026Analytical Provisioning for Attention-FFN Disaggregated LLM Serving under Stochastic Workloadscited by 2
- 2026GPU-to-Grid: Voltage Regulation via GPU Utilization Controlcited by 2
- 2026inference-fleet-sim: A Queueing-Theory-Grounded Fleet Capacity Planner for LLM Inferencecited by 2
- 2025Mitigation of Datacenter Demand Ramping and Fluctuation using Hybrid ESS and Supercapacitorcited by 1
- 2026From Tokens to Energy Flexibility: Quantization-Enabled Demand Response for Data Centers with LLM Inference Workloadscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025FlowKV: A Disaggregated Inference Framework with Low-Latency KV Cache Transfer and Load-Aware Scheduling
- 2026When Does Disaggregation Pay? Simulating Prefill--Decode--Attention--FFN Specialization for Agentic LLM Inference
- 2026Optimizing AI Inference Across the Deployment Stack
- 2025BanaServe: Unified KV Cache and Dynamic Module Migration for Balancing Disaggregated LLM Serving in AI Infrastructure
- 2026Heterogeneous Computing: The Key to Powering the Future of AI Agent Inference
- 2025DOPD: A Dynamic PD-Disaggregation Architecture for Maximizing Goodput in LLM Inference Serving
- 2026DualPath: Breaking the Storage Bandwidth Bottleneck in Agentic LLM Inference
- 2025SpecOffload: Unlocking Latent GPU Capacity for LLM Inference on Resource-Constrained Devices