mlsanityData to 14 Oct 2026

Injecting Adrenaline into LLM Serving: Boosting Resource Utilization and Throughput via Attention Disaggregation

arXiv 2503.20552 ↗First posted 26 Mar 2025cs.DC cs.LGcited by 14

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

In large language model (LLM) serving systems, executing each request consists of two phases: the compute-intensive prefill phase and the memory-intensive decoding phase. To prevent performance interference between the two phases, current LLM serving systems typically adopt prefill-decoding disaggregation, where the two phases are split across separate machines. However, we observe this approach leads to significant resource underutilization. Specifically, prefill instances that are compute-intensive suffer from low memory utilization, while decoding instances that are memory-intensive experience low compute utilization. To address this problem, this paper proposes Adrenaline, an attention disaggregation and offloading mechanism designed to enhance resource utilization and performance in LLM serving systems. Adrenaline's key innovation lies in disaggregating part of the attention computation in the decoding phase and offloading them to prefill instances. The memory-bound nature of decoding-phase attention computation inherently enables an effective offloading strategy, yielding two complementary advantages: 1) improved memory capacity and bandwidth utilization in prefill instances, and 2) increased decoding batch sizes that enhance compute utilization in decoding instances, collectively boosting overall system performance. Adrenaline achieves these gains through three key techniques: low-latency decoding synchronization, resource-efficient prefill colocation, and load-aware offloading scheduling. Experimental results show that Adrenaline achieves 2.28x higher memory capacity and 2.07x better memory bandwidth utilization in prefill instances, up to 1.67x improvements in compute utilization for decoding instances, and 1.68x higher overall inference throughput compared to state-of-the-art systems.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 13 of its 22 references that are in the map. First posted on arXiv 26 Mar 2025.

Builds on

13 of 22 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024NEO: Saving GPU Memory Crisis with CPU Offloading for Online LLM Inference2 Nov
  2. 2024Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving24 Jun
  3. 2024FastDecode: High-Throughput GPU-Efficient LLM Serving using Heterogeneous Pipelines18 Mar
  4. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving18 Jan

Other references · 9

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  6. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  7. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 558
  8. 2024Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttentioncited by 163
  9. 2024Efficient Heterogeneous Large Language Model Decoding with Model-Attention Disaggregationcited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 8; 2026: 6.

14 papers.

Sort:

By area:

  1. 2025Prefill-Decode Aggregation or Disaggregation? Unifying Both for Goodput-Optimized LLM Servingcited by 23
  2. 2025PDTrim: Targeted Pruning for Prefill-Decode Disaggregation in Inferencecited by 14
  3. 2025DynaServe: Unified and Elastic Execution for Dynamic Disaggregated LLM Servingcited by 6
  4. 2025FairBatching: Fairness-Aware Batch Formation for LLM Inferencecited by 6
  5. 2025Arrow: Adaptive Scheduling Mechanisms for Disaggregated LLM Inference Architecturecited by 5
  6. 2025Toward Cost-Efficient Serving of Mixture-of-Experts with Asynchronycited by 3
  7. 2025Disaggregated Prefill and Decoding Inference System for Large Language Model Serving on Multi-Vendor GPUscited by 3
  8. 2026RelayGR: Scaling Long-Sequence Generative Recommendation via Cross-Stage Relay-Race Inferencecited by 2
  9. 2026Coordinated Scheduling for MoE LLM Servingcited by 2
  10. 2025Efficient MoE Inference with Fine-Grained Scheduling of Disaggregated Expert Parallelismcited by 1
  11. 2026Move the Query, Not the Cache: Characterizing Cross-Instance Latent Attention Redistribution Across GPU Fabricscited by 1
  12. 2026Language Models Can Control Their Own Attentioncited by 1
2 more
  1. 2026Q-First: Most of Attention Needs Only the Query in Disaggregated LLM Decodingcited by 0
  2. 2026LLM Inference on IMC-NoC Architecture with Balanced Dataflow and Fine-Grained Parallelismcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024ScaleLLM: A Resource-Frugal LLM Serving Framework by Optimizing End-to-End Efficiency
  2. 2026How Far Can Disaggregation Go? A Design-Space Exploration of Attention-FFN Disaggregation for Efficient MoE LLM Serving
  3. 2025VoltanaLLM: Energy-Efficient and SLO-Aware Disaggregated LLM Serving via Adaptive Frequency Control and State-Space Routing
  4. 2026Analytical Provisioning for Attention-FFN Disaggregated LLM Serving under Stochastic Workloads
  5. 2024Infinite-LLM: Efficient LLM Service for Long Context with DistAttention and Distributed KVCache
  6. 2025DOPD: A Dynamic PD-Disaggregation Architecture for Maximizing Goodput in LLM Inference Serving
  7. 2025BanaServe: Unified KV Cache and Dynamic Module Migration for Balancing Disaggregated LLM Serving in AI Infrastructure
  8. 2025PAT: Accelerating LLM Decoding via Prefix-Aware Attention with Resource Efficient Multi-Tile Kernel