mlsanityData to 7 Oct 2026

Optimus: Elastic Decoding for Efficient Diffusion LLM Serving

arXiv 2605.24832 ↗First posted 24 May 2026cs.DCcited by 1

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

Large language model (LLM) serving is fundamentally limited by inefficient hardware utilization. Autoregressive (AR) decoding underutilizes GPUs due to its strictly sequential execution, while diffusion LLMs (DLLMs) improve throughput by decoding multiple tokens per iteration. However, fixed block-size diffusion decoding exhibits strong load sensitivity: large blocks exploit idle GPU resources under low load, but saturate early and incur substantial redundant computation under high load. As a result, throughput gains vanish beyond saturation, and no single decoding granularity performs well across dynamic serving workloads. We present Optimus, a serving system that enables elastic decoding for diffusion LLMs by dynamically adapting decoding granularity to runtime load. The key idea is to treat decoding granularity as a runtime control variable, balancing GPU utilization and token efficiency. Optimus combines chunked decoding, which enables fine-grained execution without retraining, with saturation-aware scheduling, a closed-loop mechanism that selects chunk sizes based on runtime conditions. Together with system-level optimizations and customized attention kernels, Optimus achieves significant performance improvements while preserving model accuracy. Experiments show that Optimus delivers up to 6.1x throughput improvement over AR decoding and 4.3x improvement over fixed-block diffusion LLM, while maintaining stable performance across diverse load regimes and improving end-to-end serving capacity under latency constraints. The source code is available at https://github.com/dubcyfor3/Optimus.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 32 of its 50 references that are in the map. First posted on arXiv 24 May 2026.

Builds on

32 of 50 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving18 Jan

Other references · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2025Qwen3 Technical Reportcited by 8,195
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  7. 2021Program Synthesis with Large Language Modelscited by 3,509
  8. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  9. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  10. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  11. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
  12. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
19 more
  1. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  2. 2025Large Language Diffusion Modelscited by 872
  3. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 700
  4. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertaintycited by 564
  5. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
  6. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Testcited by 318
  7. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 273
  8. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 168
  9. 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100Bcited by 154
  10. 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generationcited by 118
  11. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcingcited by 111
  12. 2025FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusioncited by 77
  13. 2025Diffusion Language Models Know the Answer Before Decodingcited by 68
  14. 2025Evaluating LLM-based Agents for Multi-Turn Conversations: A Surveycited by 56
  15. 2025Towards Greater Leverage: Scaling Laws for Efficient Mixture-of-Experts Language Modelscited by 44
  16. 2025DPad: Efficient Diffusion Language Models with Suffix Dropoutcited by 38
  17. 2025WeDLM: Reconciling Diffusion Language Models with Standard Causal Attention for Fast Inferencecited by 38
  18. 2024TurboSpec: Closed-loop Speculation Control System for Optimizing LLM Serving Goodputcited by 36
  19. 2025AdaSPEC: Selective Knowledge Distillation for Efficient Speculative Decoderscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Denoising Surface: Modeling and Predicting Inference Cost for Diffusion LLM Servingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024FastDecode: High-Throughput GPU-Efficient LLM Serving using Heterogeneous Pipelines
  2. 2023The Synergy of Speculative Decoding and Batching in Serving Large Language Models
  3. 2024Break the Sequential Dependency of LLM Inference Using Lookahead Decoding
  4. 2026The Economics of AI Decoding Chips: Rebalancing Compute, Capacity, and Bandwidth for Efficient LLM Inference
  5. 2023Efficient LLM inference solution on Intel GPU
  6. 2025Accelerating Diffusion LLMs via Adaptive Parallel Decoding
  7. 2024Dovetail: A CPU/GPU Heterogeneous Speculative Decoding for LLM inference
  8. 2025Helix Parallelism: Rethinking Sharding Strategies for Interactive Multi-Million-Token LLM Decoding