How Far Can Disaggregation Go? A Design-Space Exploration of Attention-FFN Disaggregation for Efficient MoE LLM Serving
On the map: Mixture-of-experts → Efficient MoE training and inference systems
Abstract (arXiv)
Modern large language model (LLM) inference has progressively disaggregated to keep pace with growing model sizes and tight TTFT and TPOT service-level objectives: from chunked-prefill aggregation, to prefill-decode (P/D) disaggregation, and most recently to operator-level Attention-FFN Disaggregation (AFD). This trend is especially important for mixture-of-experts (MoE) models, where memory-bound attention, compute-intensive expert FFNs, and MoE dispatch/combine communication create distinct resource demands. AFD further exposes this heterogeneity by placing attention and MoE-FFN execution on separate GPU groups. Each level of disaggregation deepens the scheduling design space across workload characteristics, resource allocation, and interconnect topology, raising the central question: when does each level actually pay off? We systematically characterize this trade-off for MoE inference across realistic workloads spanning input/output sequence lengths, prefix-KV reuse, and per-user latency constraints. Using chunked-prefill and P/D disaggregation as baselines, we study the benefits and limits of AFD at scale through a framework that fuses on-device kernel measurements with high-fidelity network simulation. Under strict TTFT/TPOT SLOs, AFD sustains around 4k tokens/s of system throughput on DeepSeek-V3.2 across chat, coding, and agentic-coding workloads, where non-AFD deployments are infeasible. We distill concrete takeaways for jointly optimizing throughput and interactivity, including how to partition attention and FFN across GPUs as a function of workload and model architecture, providing design principles for current rack- and cluster-scale deployments as well as future disaggregated AI infrastructure.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Efficient MoE training and inference systems, one of 12 directions in Mixture-of-experts. Builds on 10 of its 21 references that are in the map. First posted on arXiv 27 May 2026.
Builds on
10 of 21 references.
Closest ancestors · 1
Same direction, newest first
Other references · 9
- 2017Attention Is All You Needcited by 60,361
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
- 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 617
- 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 558
- 2023SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefillscited by 193
- 2024Vidur: A Large-Scale Simulation Framework For LLM Inferencecited by 117
- 2022Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelismcited by 53
- 2026AIConfigurator: Lightning-Fast Configuration Optimization for Multi-Framework LLM Servingcited by 9
- 2025MIST: A Co-Design Framework for Heterogeneous, Multi-Stage LLM Inferencecited by 2
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Revealing the Challenges of Attention-FFN Disaggregation for Modern MoE Models and Hardware Systems
- 2026Beyond Prefill-Decode Disaggregation: Dissecting LLM Inference for Heterogeneous Platforms via Dynamic Operator Scheduling
- 2026Analytical Provisioning for Attention-FFN Disaggregated LLM Serving under Stochastic Workloads
- 2025Injecting Adrenaline into LLM Serving: Boosting Resource Utilization and Throughput via Attention Disaggregation
- 2025DOPD: A Dynamic PD-Disaggregation Architecture for Maximizing Goodput in LLM Inference Serving
- 2024Efficient Heterogeneous Large Language Model Decoding with Model-Attention Disaggregation
- 2026SLO-Aware Compute Resource Allocation for Prefill-Decode Disaggregated LLM Inference
- 2026Rethinking Heterogeneous System Disaggregation for Subquadratic Attention