mlsanityData to 7 Oct 2026

Revisiting Disaggregated Large Language Model Serving for Performance and Energy Implications

arXiv 2601.08833 ↗First posted 14 Nov 2025cs.PF cs.AI cs.ARcited by 2

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

Different from traditional Large Language Model (LLM) serving that colocates the prefill and decode stages on the same GPU, disaggregated serving dedicates distinct GPUs to prefill and decode workload. Once the prefill GPU completes its task, the KV cache must be transferred to the decode GPU. While existing works have proposed various KV cache transfer paths across different memory and storage tiers, there remains a lack of systematic benchmarking that compares their performance and energy efficiency. Meanwhile, although optimization techniques such as KV cache reuse and frequency scaling have been utilized for disaggregated serving, their performance and energy implications have not been rigorously benchmarked. In this paper, we fill this research gap by re-evaluating prefill-decode disaggregation under different KV transfer mediums and optimization strategies. Specifically, we include a new colocated serving baseline and evaluate disaggregated setups under different KV cache transfer paths. Through GPU profiling using dynamic voltage and frequency scaling (DVFS), we identify and compare the performance-energy Pareto frontiers across all setups to evaluate the potential energy savings enabled by disaggregation. Our results show that performance benefits from prefill-decode disaggregation are not guaranteed and depend on the request load and KV transfer mediums. In addition, stage-wise independent frequency scaling enabled by disaggregation does not lead to energy saving due to inherently higher energy consumption of disaggregated serving.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 34 of its 62 references that are in the map. First posted on arXiv 14 Nov 2025.

Builds on

34 of 62 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Prefill-Decode Aggregation or Disaggregation? Unifying Both for Goodput-Optimized LLM Serving4 Aug
  2. 2025semi-PD: Towards Efficient LLM Serving via Phase-Wise Disaggregated Computation and Unified Storage28 Apr
  3. 2024GreenLLM: Disaggregating Large Language Model Serving on Heterogeneous GPUs for Lower Carbon Emissions29 Dec
  4. 2024SLO-aware GPU Frequency Scaling for Energy Efficient LLM Inference Serving5 Aug
  5. 2024Offline Energy-Optimal LLM Serving: Workload-Based Energy Models for LLM Inference on Heterogeneous Systems4 Jul
  6. 2024Andes: Defining and Enhancing Quality-of-Experience in LLM-Based Text Streaming Services25 Apr
  7. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving18 Jan

Other references · 27

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  5. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  6. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 536
  7. 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 255
  8. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusioncited by 211
  9. 2023CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Servingcited by 199
  10. 2024Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttentioncited by 159
  11. 2024DynamoLLM: Designing LLM Inference Clusters for Performance and Energy Efficiencycited by 106
  12. 2024A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methodscited by 101
15 more
  1. 2024RAGCache: Efficient Knowledge Caching for Retrieval-Augmented Generationcited by 97
  2. 2023Stateful Large Language Model Serving with Pensievecited by 59
  3. 2024Towards Greener LLMs: Bringing Energy-Efficiency to the Forefront of LLM Inferencecited by 59
  4. 2023Bias Testing and Mitigation in LLM-based Code Generationcited by 53
  5. 2024LLM-Based Test-Driven Interactive Code Generation: User Study and Empirical Evaluationcited by 46
  6. 2024EPIC: Efficient Position-Independent Caching for Serving Large Language Modelscited by 45
  7. 2023Summarization is (Almost) Deadcited by 43
  8. 2025KVCache Cache in the Wild: Characterizing and Optimizing KVCache Cache at a Large Cloud Providercited by 41
  9. 2025Sustainable LLM Inference for Edge AI: Evaluating Quantized LLMs for Energy Efficiency, Output Accuracy, and Inference Latencycited by 32
  10. 2024LLM-based Conversational AI Therapist for Daily Functioning Screening and Psychotherapeutic Intervention via Everyday Smart Devicescited by 21
  11. 2024Hybrid Heterogeneous Clusters Can Lower the Energy Consumption of LLM Inference Workloadscited by 17
  12. 2023User Interaction Patterns and Breakdowns in Conversing with LLM-Powered Voice Assistantscited by 14
  13. 2025TD-Pipe: Temporally-Disaggregated Pipeline Parallelism Architecture for High-Throughput LLM Inferencecited by 8
  14. 2025Taming the Chaos: Coordinated Autoscaling for Heterogeneous and Disaggregated LLM Inferencecited by 5
  15. 2025EcoServe: Enabling Cost-effective LLM Serving with Proactive Intra- and Inter-Instance Orchestrationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Simple is Better: Multiplication May Be All You Need for LLM Request Schedulingcited by 2
  2. 2026Cascade: Exploiting SLO-Aware latency budget for fair and high goodput LLM inference servingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024KVDirect: Distributed Disaggregated LLM Inference
  2. 2025Nexus:Proactive Intra-GPU Disaggregation of Prefill and Decode in LLM Serving
  3. 2025Mell: Memory-Efficient Large Language Model Serving via Multi-GPU KV Cache Management
  4. 2024ConServe: Fine-Grained GPU Harvesting for LLM Online and Offline Co-Serving
  5. 2024LayerKV: Optimizing Large Language Model Serving with Layer-wise KV Cache Management
  6. 2025WarmServe: Enabling One-for-Many GPU Prewarming for Multi-LLM Serving
  7. 2025TraCT: Disaggregated LLM Serving with CXL Shared Memory KV Cache at Rack-Scale
  8. 2024FastDecode: High-Throughput GPU-Efficient LLM Serving using Heterogeneous Pipelines