mlsanityData to 7 Oct 2026

SLO-Aware Compute Resource Allocation for Prefill-Decode Disaggregated LLM Inference

arXiv 2603.04716 ↗First posted 5 Mar 2026cs.DC cs.IT cs.LGcited by 1

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

Prefill-Decode (P/D) disaggregation has emerged as a widely adopted optimization strategy for Large Language Model (LLM) inference. However, there currently exists no well-established methodology for determining the optimal number of P/D hardware resources, subject to constraints on total throughput, service level objectives (SLOs), and request characteristics - specifically input and output lengths. To address this gap, we propose a hybrid approach that combines theoretical modeling with empirical benchmarking. First, we present a theoretical model for calculating P/D resource counts, which is based on total throughput requirements, request input and output lengths, as well as prefill and decode throughput. Then, to obtain the actual prefill and decode throughput under SLO constraints, we model the prefill process using M/M/1 queuing theory, deriving the achieved prefill throughput from the benchmarked maximum prefill throughput and Time-To-First-Token (TTFT). For the decode phase, we determine the decode batch sizes that meet Time-Per-Output-Token (TPOT) requirements and obtain the corresponding decode throughput through empirical measurements. Our experimental results demonstrate that the proposed method can accurately predict optimal P/D resource allocation in real-world LLM inference scenarios.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 16 of its 22 references that are in the map. First posted on arXiv 5 Mar 2026.

Builds on

16 of 22 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026AIConfigurator: Lightning-Fast Configuration Optimization for Multi-Framework LLM Serving9 Jan
  2. 2025Disaggregated Prefill and Decoding Inference System for Large Language Model Serving on Multi-Vendor GPUs22 Sep
  3. 2024P/D-Serve: Serving Disaggregated Large Language Model at Scale15 Aug
  4. 2024MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool25 Jun
  5. 2024Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving24 Jun
  6. 2023SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills31 Aug

Other references · 10

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  3. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  4. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 536
  5. 2025LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inferencecited by 119
  6. 2025KVFlow: Efficient Prefix Caching for Accelerating LLM-Based Multi-Agent Workflowscited by 56
  7. 2025Strata: Hierarchical Context Caching for Long Context Language Model Servingcited by 25
  8. 2024Efficiently Serving Large Multimodal Models Using EPD Disaggregationcited by 16
  9. 2025DOPD: A Dynamic PD-Disaggregation Architecture for Maximizing Goodput in LLM Inference Servingcited by 5
  10. 2025STAR: Decode-Phase Rescheduling for LLM Inferencecited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Analytical Power-Aware Provisioning for Prefill-Decode Disaggregated AI Inferencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Optimal Scheduling Algorithms for LLM Inference: Theory and Practice
  2. 2025SPAD: Specialized Prefill and Decode Hardware for Disaggregated LLM Inference
  3. 2025VoltanaLLM: Energy-Efficient and SLO-Aware Disaggregated LLM Serving via Adaptive Frequency Control and State-Space Routing
  4. 2025Arrow: Adaptive Scheduling Mechanisms for Disaggregated LLM Inference Architecture
  5. 2026Taming Request Imbalance: SLO-Aware Scheduling for Disaggregated LLM Inference ↗
  6. 2026Beyond Prefill-Decode Disaggregation: Dissecting LLM Inference for Heterogeneous Platforms via Dynamic Operator Scheduling
  7. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving
  8. 2025AccelGen: Heterogeneous SLO-Guaranteed High-Throughput LLM Inference Serving for Diverse Applications