# AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters

arXiv: https://arxiv.org/abs/2607.19223 · HTML: https://mlsanity.com/paper/2607.19223/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving.
- Builds on 31 of its 68 references that are in the map.
- First posted on arXiv 21 Jul 2026.

## Placed in

- Efficient LLM inference and serving / [Speculative decoding](https://mlsanity.com/inference/speculative-decoding/)

## Builds on (most cited first, 30 of 31)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [Accelerating Large Language Model Decoding with Speculative Sampling](https://mlsanity.com/paper/2302.01318/) (2023)
- [DeepSeek-V3 Technical Report](https://mlsanity.com/paper/2412.19437/) (2024)
- [Dream 7B: Diffusion Large Language Models](https://mlsanity.com/paper/2508.15487/) (2025)
- [Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding](https://mlsanity.com/paper/2505.22618/) (2025)
- [EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test](https://mlsanity.com/paper/2503.01840/) (2025)
- [LLaDA2.0: Scaling Up Diffusion Language Models to 100B](https://mlsanity.com/paper/2512.15745/) (2025)
- [Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing](https://mlsanity.com/paper/2508.09192/) (2025)
- [Revolutionizing Reinforcement Learning Framework for Diffusion Large Language Models](https://mlsanity.com/paper/2509.06949/) (2025)
- [dParallel: Learnable Parallel Decoding for dLLMs](https://mlsanity.com/paper/2509.26488/) (2025)
- [OpenCodeInstruct: A Large-scale Instruction Tuning Dataset for Code LLMs](https://mlsanity.com/paper/2504.04030/) (2025)
- [DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation](https://mlsanity.com/paper/2607.05147/) (2026)
- [dInfer: An Efficient Inference Framework for Diffusion Language Models](https://mlsanity.com/paper/2510.08666/) (2025)
- [d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillation](https://mlsanity.com/paper/2601.07568/) (2026)
- [Efficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speed](https://mlsanity.com/paper/2512.14067/) (2025)
- [Multi-Candidate Speculative Decoding](https://mlsanity.com/paper/2401.06706/) (2024)
- [Self Speculative Decoding for Diffusion Large Language Models](https://mlsanity.com/paper/2510.04147/) (2025)
- [AdaEAGLE: Optimizing Speculative Decoding via Explicit Modeling of Adaptive Draft Structures](https://mlsanity.com/paper/2412.18910/) (2024)
- [ReFusion: A Diffusion Large Language Model with Parallel Autoregressive Decoding](https://mlsanity.com/paper/2512.13586/) (2025)
- [DMax: Aggressive Parallel Decoding for dLLMs](https://mlsanity.com/paper/2604.08302/) (2026)
- [Taming Masked Diffusion Language Models via Consistency Trajectory Reinforcement Learning with Fewer Decoding Step](https://mlsanity.com/paper/2509.23924/) (2025)
- [JetSpec: Breaking the Scaling Ceiling of Speculative Decoding with Parallel Tree Drafting](https://mlsanity.com/paper/2606.18394/) (2026)
- [Fast and Accurate Causal Parallel Decoding using Jacobi Forcing](https://mlsanity.com/paper/2512.14681/) (2025)
- [Draft-OPD: On-Policy Distillation for Speculative Draft Models](https://mlsanity.com/paper/2605.29343/) (2026)
- [Not-a-Bandit: Provably No-Regret Drafter Selection in Speculative Decoding for LLMs](https://mlsanity.com/paper/2510.20064/) (2025)
- [AngelSlim: A more accessible, comprehensive, and efficient toolkit for large model compression](https://mlsanity.com/paper/2602.21233/) (2026)
- [Fail Fast, Win Big: Rethinking the Drafting Strategy in Speculative Decoding via Diffusion LLMs](https://mlsanity.com/paper/2512.20573/) (2025)

## Built on by (most cited first, 4 of 4)

- [LongSpark: Efficient speculative decoding with a fixed-cost parallel drafter](https://mlsanity.com/paper/2609.37029/) (2026)
- [STEPQuant: When and Where Errors Matter in Delta-Rule Recurrent State Quantization](https://mlsanity.com/paper/2609.38169/) (2026)
- [CAST: Cost-Aware Speculative Trees from One-Pass Block Drafters](https://mlsanity.com/paper/2610.00321/) (2026)
- [DRelay: Global Draft Context for Prefix-Aware Parallel Speculative Decoding Repair](https://mlsanity.com/paper/2610.01439/) (2026)

## Abstract (arXiv)

Speculative decoding, in which a lightweight draft model first generates a draft sequence that is then verified by the target model, has become a prevalent paradigm for accelerating large language model inference. Recent work such as DFlash further boosts drafting efficiency by leveraging diffusion drafters, whose parallel denoising mechanism enables draft generation in a single forward pass. In this work, we uncover a central pitfall of diffusion drafters: the global dependency, arising from bidirectional attention and KV injection in diffusion drafters, is a double-edged sword. On one hand, it endows the model with parallel generation and global contextual modeling capabilities; on the other hand, this global dependency introduces high variance at both the domain-level and the token-level: acceptance rates fluctuate substantially across different domains, and the acceptance probability varies across draft positions. To tackle this issue, we propose the AdaFlash framework, comprising two components: (i) an on-policy distillation (OPD) algorithm with reverse-KL divergence tailored for diffusion drafters, which delivers stable convergence and continuously adapts the drafter to the deployment distribution; and (ii) an adaptive length head that dynamically adjusts the candidate length on the fly, substantially lowering the verification cost of the target model. Experiments demonstrate that AdaFlash consistently improves the speedup during deployment, with especially significant gains under high-concurrency, achieving up to 66% higher average throughput than previous state-of-the-art methods. Our code is available at https://github.com/ZinYY/AdaFlash.