mlsanityData to 14 Oct 2026

Analytical Provisioning for Attention-FFN Disaggregated LLM Serving under Stochastic Workloads

arXiv 2601.21351 ↗First posted 29 Jan 2026cs.LG cs.AIcited by 2

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

Attentio-FFN disaggregation (AFD) is an emerging architecture for LLM decoding that separates state-heavy, KV-cache-dominated Attention computation from stateless, compute-intensive FFN computation, connected by per-step communication. While AFD enables independent scaling of memory and compute resources, its performance is highly sensitive to the Attention/FFN provisioning ratio: mis-sizing induces step-level blocking and costly device idle time. We develop an analytical provisioning framework for AFD bundles in an rA--1F topology under stochastic workloads. Two sources of randomness shape the problem: per-slot Attention workload evolves as KV caches grow and completed requests are replenished with random prompt and decode lengths, and synchronized execution across Attention workers introduces a barrier governed by the slowest worker. We address both via a renewal-reward characterization of the per-slot stationary token load, identifying a single workload statistic θ that governs provisioning under arbitrary prefill-decode distributions and admits a nonparametric estimator from request traces. The analysis yields a closed-form mean-field rule for the optimal A/F ratio decomposing into Attention-, communication-, and FFN-bottleneck regimes, together with a Gaussian barrier-aware refinement that quantifies cross-worker synchronization overhead. A trace-calibrated AFD simulator supports the framework across workloads: the predicted optimal ratio matches the simulation-optimal within 10%. Together, these results provide a compact, calibratable account of how stochastic workload structure determines provisioning in disaggregated LLM serving.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 21 of its 24 references that are in the map. First posted on arXiv 29 Jan 2026.

Builds on

21 of 24 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttention12 Sep

Other references · 20

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2020Scaling Laws for Neural Language Modelscited by 6,586
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  5. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 629
  6. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 617
  7. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 558
  8. 2022Efficiently Scaling Transformer Inferencecited by 539
  9. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Servecited by 526
  10. 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 461
  11. 2023OpenChat: Advancing Open-source Language Models with Mixed-Quality Datacited by 252
  12. 2024LLM Inference Unveiled: Survey and Roofline Model Insightscited by 152
8 more
  1. 2024BurstGPT: A Real-world Workload Dataset to Optimize LLM Serving Systemscited by 87
  2. 2025Step-3 is Large yet Affordable: Model-system Co-design for Cost-effective Decodingcited by 55
  3. 2025MegaScale-Infer: Serving Mixture-of-Experts at Scale with Disaggregated Expert Parallelismcited by 32
  4. 2025Serving Large Language Models on Huawei CloudMatrix384cited by 24
  5. 2024Flash Communication: Reducing Tensor Parallelization Bottleneck for Fast Large Language Model Inferencecited by 11
  6. 2025Janus: Disaggregating Attention and Experts for Scalable MoE Inferencecited by 8
  7. 2026A Universal Load Balancing Principle and Its Application to Large Language Model Servingcited by 8
  8. 2025Boosting Performance of Iterative Applications on GPUs: Kernel Batching with CUDA Graphscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026OpWeave: Flexible Operator Disaggregation for Heterogeneous LLM Servingcited by 0
  2. 2026Analytical Power-Aware Provisioning for Prefill-Decode Disaggregated AI Inferencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026How Far Can Disaggregation Go? A Design-Space Exploration of Attention-FFN Disaggregation for Efficient MoE LLM Serving
  2. 2025Injecting Adrenaline into LLM Serving: Boosting Resource Utilization and Throughput via Attention Disaggregation
  3. 2026Rethinking Heterogeneous System Disaggregation for Subquadratic Attention
  4. 2026Q-First: Most of Attention Needs Only the Query in Disaggregated LLM Decoding
  5. 2025PAT: Accelerating LLM Decoding via Prefix-Aware Attention with Resource Efficient Multi-Tile Kernel
  6. 2026Revealing the Challenges of Attention-FFN Disaggregation for Modern MoE Models and Hardware Systems
  7. 2026Energy-Efficient LLM Serving via Disaggregated Attention--FFN and Flexible Frequency Scaling
  8. 2026SLO-Aware Compute Resource Allocation for Prefill-Decode Disaggregated LLM Inference