mlsanityData to 7 Oct 2026

Prefill-Decode Aggregation or Disaggregation? Unifying Both for Goodput-Optimized LLM Serving

arXiv 2508.01989 ↗First posted 4 Aug 2025cs.DCcited by 21

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

An ongoing debate considers whether prefill-decode (PD) aggregation or disaggregation is superior for serving large language models (LLMs). This has driven optimizations for both approaches, each showing distinct advantages. This paper compares PD aggregation and disaggregation, showing that each excels under different service-level objectives (SLOs): aggregation is optimal for tight time-to-first-token (TTFT) and relaxed time-per-output-token (TPOT), while disaggregation excels for strict TPOT and relaxed TTFT. However, under balanced TTFT and TPOT SLOs, neither approach delivers optimal goodput. This paper proposes TaiChi, an LLM serving system that unifies PD disaggregation and aggregation for optimal goodput under any combination of TTFT and TPOT SLOs. TaiChi uses a unified disaggregation-aggregation architecture with differentiated-capability GPU instances: prefill-heavy (fast prefill, high-interference decode) and decode-heavy (low-interference decode, slow prefill). Three configurable sliders control the ratio between these instances and their chunk sizes. TaiChi adapts to various SLO regimes by adjusting sliders. When TTFT constraints are tight, TaiChi resembles a PD aggregation configuration; when TPOT dominates, it adapts toward PD disaggregation. Crucially, under balanced SLOs, TaiChi enables a hybrid mode for superior goodput. The key innovation behind this hybrid mode is latency shifting: selectively reallocating GPU resources from requests that meet SLOs to those at risk of violation, maximizing the number of SLO-satisfied requests. This fine-grained latency shifting is orchestrated by two scheduling mechanisms: flowing decode scheduling to control TPOTs and length-aware prefill scheduling to manage TTFTs, which jointly optimize request assignment. Our experiments show TaiChi improves goodput by up to 77% over state-of-the-art systems under balanced TTFT and TPOT SLOs.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 16 of its 34 references that are in the map. First posted on arXiv 4 Aug 2025.

Builds on

16 of 34 references.

Closest ancestors · 10

Same direction, newest first

  1. 2025JITServe: SLO-aware LLM Serving with Imprecise Request Information24 Apr
  2. 2025Injecting Adrenaline into LLM Serving: Boosting Resource Utilization and Throughput via Attention Disaggregation26 Mar
  3. 2024Revisiting Service Level Objectives and System Level Metrics in Large Language Model Serving18 Oct
  4. 2024NanoFlow: Towards Optimal Large Language Model Serving Throughput22 Aug
  5. 2024Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving24 Jun
  6. 2024Efficient Interactive LLM Serving with Proxy Model-based Sequence Length Prediction12 Apr
  7. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve4 Mar
  8. 2024InferCept: Efficient Intercept Support for Augmented Large Language Model Inference2 Feb
  9. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving18 Jan
  10. 2023Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline22 May

Other references · 6

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  4. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  5. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 536
  6. 2023S3: Increasing GPU Utilization during Generative Inference for Higher Throughputcited by 81

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 8; 2026: 13.

21 papers.

Sort:

By area:

  1. 2025PDTrim: Targeted Pruning for Prefill-Decode Disaggregation in Inferencecited by 14
  2. 2025FairBatching: Fairness-Aware Batch Formation for LLM Inferencecited by 6
  3. 2025TokenLake: A Unified Segment-level Prefix Cache Pool for Fine-grained Elastic Long-Context LLM Servingcited by 5
  4. 2025From Tokens to Layers: Redefining Stall-Free Scheduling for MoE Serving with Layered Prefillcited by 3
  5. 2025Astrolabe: Balancing Load in LLM Serving with Randomized Prediction-Guided Schedulingcited by 2
  6. 2025HFX: Joint Design of Algorithms and Systems for Multi-SLO Serving and Fast Scalingcited by 2
  7. 2026RelayGR: Scaling Long-Sequence Generative Recommendation via Cross-Stage Relay-Race Inferencecited by 2
  8. 2025Revisiting Disaggregated Large Language Model Serving for Performance and Energy Implicationscited by 2
  9. 2026Observation, Not Prediction: Conversation-Level Disaggregated Scheduling for Agentic Servingcited by 2
  10. 2026Coordinated Scheduling for MoE LLM Servingcited by 2
  11. 2026Towards Load-Aware Prefill Deflection for Disaggregated LLM Servingcited by 2
  12. 2026SMetric: Rethink LLM Scheduling for Serving Agents with Balanced Session-centric Schedulingcited by 2
9 more
  1. 2026The Price of Anarchy in Disaggregated Inferencecited by 1
  2. 2026ExpertPlex: A High-Goodput Disaggregated Serving System for MoE LLMs with Adaptive Persistent Kernelscited by 1
  3. 2025ProServe: Unified Multi-Priority Request Scheduling for LLM Servingcited by 0
  4. 2026Balancing Sustainability And Performance: The Role Of Small-Scale LLMs In Agentic Artificial Intelligence Systemscited by 0
  5. 2026LLM Inference on IMC-NoC Architecture with Balanced Dataflow and Fine-Grained Parallelismcited by 0
  6. 2026Toward Sustainable Distributed LLM Inference: A Systems Synthesis and Research Agenda for an Energy-, Carbon-, and Cache-Aware llm-d Control Planecited by 0
  7. 2026PipeSwift: Revisiting Pipeline Parallelism for Large-Scale Completion-Oriented Agentic LLM Servingcited by 0
  8. 2026Crossflow: Prefill-Decode Elasticity for Agentic LLM Servingcited by 0
  9. 2026FluidPD: In-Place Elasticity for SLO-Aware Prefill-Decode Disaggregated LLM Servingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026SLO-Aware Compute Resource Allocation for Prefill-Decode Disaggregated LLM Inference
  2. 2025DOPD: A Dynamic PD-Disaggregation Architecture for Maximizing Goodput in LLM Inference Serving
  3. 2025SPAD: Specialized Prefill and Decode Hardware for Disaggregated LLM Inference
  4. 2026PDD: Unleashing Economical and Flexible Heterogeneous LLM Inference via Cross-Datacenter Prefill-Decode Disaggregation
  5. 2026Tropical: Enhancing SLO Attainment in Disaggregated LLM Serving via SLO-Aware Multiplexing
  6. 2025Trinity: Disaggregating Vector Search from Prefill-Decode Disaggregation in LLM Serving
  7. 2026How Far Can Disaggregation Go? A Design-Space Exploration of Attention-FFN Disaggregation for Efficient MoE LLM Serving
  8. 2026PrefillShare: A Shared Prefill Module for KV Reuse in Multi-LLM Disaggregated Serving