mlsanityData to 14 Oct 2026

Energy-Efficient LLM Serving via Disaggregated Attention--FFN and Flexible Frequency Scaling

arXiv 2608.01891 ↗First posted 3 Aug 2026cs.DC cs.AIcited by 0

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

Large language model (LLM) serving spans diverse applications with stringent service-level objectives (SLOs), often requiring GPUs to run at maximum frequencies and increasing energy consumption. Existing energy-management approaches adapt GPU frequencies only at the request or inference-phase level, overlooking operator-level differences in frequency sensitivity between Attention and feed-forward networks (FFNs). We find that the energy-optimal frequencies of Attention and FFN (A/F) differ and vary with the inference phase, workload, and system configurations. However, runtime variability and independent A/F frequency control create a large search space and high communication overhead. To address these challenges, we present AFlex, a framework that jointly optimizes resource provisioning and GPU frequency scaling for disaggregated A/F serving. AFlex introduces a global scheduler and a local operator-level dynamic voltage and frequency scaling (DVFS) controller to determine A/F resource allocations and frequencies. It further introduces an interleaved A/F pipeline with dynamic microbatch depth and adaptive request batching to reduce pipeline bubbles. We implement AFlex in SGLang and evaluate it on NVIDIA A800 GPUs using Qwen3-32B and Mixtral-8×7B under production Conversation and Coding traces. AFlex reduces energy per token by up to 49% over state-of-the-art disaggregated serving and 48% over frequency-scaling systems while satisfying TTFT and TPOT SLOs.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 21 of its 36 references that are in the map. First posted on arXiv 3 Aug 2026.

Builds on

21 of 36 references.

Closest ancestors · 9

Same direction, newest first

  1. 2026DualScale: Energy-Efficient Disaggregated LLM Serving via Phase-Aware Placement and DVFS21 Feb
  2. 2024SLO-aware GPU Frequency Scaling for Energy Efficient LLM Inference Serving5 Aug
  3. 2024MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool25 Jun
  4. 2024Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving24 Jun
  5. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve4 Mar
  6. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving18 Jan
  7. 2023FlashDecoding++: Faster Large Language Model Inference on GPUs2 Nov
  8. 2023SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills31 Aug
  9. 2023Fast Distributed Inference Serving for Large Language Models10 May

Other references · 12

  1. 2025Qwen3 Technical Reportcited by 8,820
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  3. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  4. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  5. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 575
  6. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 558
  7. 2023A Study of Generative Large Language Model for Medical Research and Healthcarecited by 163
  8. 2025LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inferencecited by 126
  9. 2024Inference without Interference: Disaggregate LLM Inference for Mixed Downstream Workloadscited by 121
  10. 2026FlashAttention-4: Algorithm and Kernel Pipelining Co-Design for Asymmetric Hardware Scalingcited by 66
  11. 2024Towards Greener LLMs: Bringing Energy-Efficiency to the Forefront of LLM Inferencecited by 61
  12. 2024Accelerating Retrieval-Augmented Generationcited by 14

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Understanding Efficiency: Quantization, Batching, and Serving Strategies in LLM Energy Use
  2. 2026Towards Multi-Model LLM Schedulers: Empirical Insights into Offloading and Preemption
  3. 2026PALS: Power-Aware LLM Serving for Mixture-of-Experts Models
  4. 2025AGFT: An Adaptive GPU Frequency Tuner for Real-Time LLM Inference Optimization
  5. 2025VoltanaLLM: Energy-Efficient and SLO-Aware Disaggregated LLM Serving via Adaptive Frequency Control and State-Space Routing
  6. 2026Watt Counts: Energy-Aware Benchmark for Sustainable LLM Inference on Heterogeneous GPU Architectures
  7. 2024Infinite-LLM: Efficient LLM Service for Long Context with DistAttention and Distributed KVCache
  8. 2025LeMix: Unified Scheduling for LLM Training and Inference on Multi-GPU Systems