mlsanityData to 14 Oct 2026

AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters

arXiv 2607.19223 ↗First posted 21 Jul 2026cs.LG cs.CLcited by 4

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

Speculative decoding, in which a lightweight draft model first generates a draft sequence that is then verified by the target model, has become a prevalent paradigm for accelerating large language model inference. Recent work such as DFlash further boosts drafting efficiency by leveraging diffusion drafters, whose parallel denoising mechanism enables draft generation in a single forward pass. In this work, we uncover a central pitfall of diffusion drafters: the global dependency, arising from bidirectional attention and KV injection in diffusion drafters, is a double-edged sword. On one hand, it endows the model with parallel generation and global contextual modeling capabilities; on the other hand, this global dependency introduces high variance at both the domain-level and the token-level: acceptance rates fluctuate substantially across different domains, and the acceptance probability varies across draft positions. To tackle this issue, we propose the AdaFlash framework, comprising two components: (i) an on-policy distillation (OPD) algorithm with reverse-KL divergence tailored for diffusion drafters, which delivers stable convergence and continuously adapts the drafter to the deployment distribution; and (ii) an adaptive length head that dynamically adjusts the candidate length on the fly, substantially lowering the verification cost of the target model. Experiments demonstrate that AdaFlash consistently improves the speedup during deployment, with especially significant gains under high-concurrency, achieving up to 66% higher average throughput than previous state-of-the-art methods. Our code is available at https://github.com/ZinYY/AdaFlash.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 31 of its 68 references that are in the map. First posted on arXiv 21 Jul 2026.

Builds on

31 of 68 references.

Closest ancestors · 11

Same direction, newest first

  1. 2026DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation6 Jul
  2. 2026JetSpec: Breaking the Scaling Ceiling of Speculative Decoding with Parallel Tree Drafting16 Jun
  3. 2026Draft-OPD: On-Policy Distillation for Speculative Draft Models28 May
  4. 2026When RL Meets Adaptive Speculative Training: A Unified Training-Serving System6 Feb
  5. 2025Fail Fast, Win Big: Rethinking the Drafting Strategy in Speculative Decoding via Diffusion LLMs23 Dec
  6. 2025Not-a-Bandit: Provably No-Regret Drafter Selection in Speculative Decoding for LLMs22 Oct
  7. 2025Self Speculative Decoding for Diffusion Large Language Models5 Oct
  8. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test3 Mar
  9. 2024AdaEAGLE: Optimizing Speculative Decoding via Explicit Modeling of Adaptive Draft Structures25 Dec
  10. 2024Multi-Candidate Speculative Decoding12 Jan
  11. 2023Accelerating Large Language Model Decoding with Speculative Sampling2 Feb

Other references · 20

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2023Let's Verify Step by Stepcited by 3,789
  5. 2024DeepSeek-V3 Technical Reportcited by 811
  6. 2025Dream 7B: Diffusion Large Language Modelscited by 544
  7. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 420
  8. 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100Bcited by 166
  9. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcingcited by 116
  10. 2025Revolutionizing Reinforcement Learning Framework for Diffusion Large Language Modelscited by 88
  11. 2025dParallel: Learnable Parallel Decoding for dLLMscited by 65
  12. 2025OpenCodeInstruct: A Large-scale Instruction Tuning Dataset for Code LLMscited by 60
8 more
  1. 2025dInfer: An Efficient Inference Framework for Diffusion Language Modelscited by 37
  2. 2026d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillationcited by 37
  3. 2025Efficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speedcited by 33
  4. 2025ReFusion: A Diffusion Large Language Model with Parallel Autoregressive Decodingcited by 18
  5. 2026DMax: Aggressive Parallel Decoding for dLLMscited by 18
  6. 2025Taming Masked Diffusion Language Models via Consistency Trajectory Reinforcement Learning with Fewer Decoding Stepcited by 17
  7. 2025Fast and Accurate Causal Parallel Decoding using Jacobi Forcingcited by 13
  8. 2026AngelSlim: A more accessible, comprehensive, and efficient toolkit for large model compressioncited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026LongSpark: Efficient speculative decoding with a fixed-cost parallel draftercited by 0
  2. 2026STEPQuant: When and Where Errors Matter in Delta-Rule Recurrent State Quantizationcited by 0
  3. 2026CAST: Cost-Aware Speculative Trees from One-Pass Block Drafterscited by 0
  4. 2026DRelay: Global Draft Context for Prefix-Aware Parallel Speculative Decoding Repaircited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026D^2SD: Accelerating Speculative Decoding with Dual Diffusion Draft Models
  2. 2025DiffuSpec: Unlocking Diffusion Language Models for Speculative Decoding
  3. 2024Speculative Diffusion Decoding: Accelerating Language Generation through Diffusion
  4. 2025SpecDiff-2: Scaling Diffusion Drafter Alignment For Faster Speculative Decoding
  5. 2026PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decoding
  6. 2026SimSD: Simple Speculative Decoding in Diffusion Language Models
  7. 2026SpecFold: Folding Multi-Branch Redundancy for Faster Speculative Decoding in Diffusion Language Models
  8. 2026Accelerating Speculative Decoding with Block Diffusion Draft Trees