# DEER: Draft with Diffusion, Verify with Autoregressive Models

arXiv: https://arxiv.org/abs/2512.15176 · HTML: https://mlsanity.com/paper/2512.15176/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 30 of its 33 references that are in the map.
- First posted on arXiv 17 Dec 2025.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)
- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)
- Efficient LLM inference and serving / [Speculative decoding](https://mlsanity.com/inference/speculative-decoding/)

## Builds on (most cited first, 30 of 30)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [Structured Denoising Diffusion Models in Discrete State-Spaces](https://mlsanity.com/paper/2107.03006/) (2021)
- [Fast Inference from Transformers via Speculative Decoding](https://mlsanity.com/paper/2211.17192/) (2022)
- [Solving Quantitative Reasoning Problems with Language Models](https://mlsanity.com/paper/2206.14858/) (2022)
- [SGLang: Efficient Execution of Structured Language Model Programs](https://mlsanity.com/paper/2312.07104/) (2023)
- [Diffusion-LM Improves Controllable Text Generation](https://mlsanity.com/paper/2205.14217/) (2022)
- [Accelerating Large Language Model Decoding with Speculative Sampling](https://mlsanity.com/paper/2302.01318/) (2023)
- [Large Language Diffusion Models](https://mlsanity.com/paper/2502.09992/) (2025)
- [Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads](https://mlsanity.com/paper/2401.10774/) (2024)
- [Enhancing Chat Language Models by Scaling High-quality Instructional Conversations](https://mlsanity.com/paper/2305.14233/) (2023)
- [EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty](https://mlsanity.com/paper/2401.15077/) (2024)
- [Dream 7B: Diffusion Large Language Models](https://mlsanity.com/paper/2508.15487/) (2025)
- [Datasets: A Community Library for Natural Language Processing](https://mlsanity.com/paper/2109.02846/) (2021)
- [Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding](https://mlsanity.com/paper/2505.22618/) (2025)
- [EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees](https://mlsanity.com/paper/2406.16858/) (2024)
- [EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test](https://mlsanity.com/paper/2503.01840/) (2025)
- [Break the Sequential Dependency of LLM Inference Using Lookahead Decoding](https://mlsanity.com/paper/2402.02057/) (2024)
- [Hydra: Sequentially-Dependent Draft Heads for Medusa Decoding](https://mlsanity.com/paper/2402.05109/) (2024)
- [OpenCodeInstruct: A Large-scale Instruction Tuning Dataset for Code LLMs](https://mlsanity.com/paper/2504.04030/) (2025)
- [Speculative Diffusion Decoding: Accelerating Language Generation through Diffusion](https://mlsanity.com/paper/2408.05636/) (2024)
- [DiffuSpec: Unlocking Diffusion Language Models for Speculative Decoding](https://mlsanity.com/paper/2510.02358/) (2025)
- [dInfer: An Efficient Inference Framework for Diffusion Language Models](https://mlsanity.com/paper/2510.08666/) (2025)
- [Self Speculative Decoding for Diffusion Large Language Models](https://mlsanity.com/paper/2510.04147/) (2025)
- [Advancing Mathematical Reasoning in Language Models: The Impact of Problem-Solving Data, Data Synthesis Methods, and Training Stages](https://mlsanity.com/paper/2501.14002/) (2025)
- [Self-Speculative Masked Diffusions](https://mlsanity.com/paper/2510.03929/) (2025)
- [INDICT: Code Generation with Internal Dialogues of Critiques for Both Security and Helpfulness](https://mlsanity.com/paper/2407.02518/) (2024)

## Built on by (most cited first, 9 of 9)

- [DART: Diffusion-Inspired Speculative Decoding for Fast LLM Inference](https://mlsanity.com/paper/2601.19278/) (2026)
- [TAPS: Target-Aware Prefix Tree Selection for Diffusion-Drafted Speculative Decoding](https://mlsanity.com/paper/2606.00487/) (2026)
- [Lookahead-then-Verify: Reliable Constrained Decoding for Diffusion LLMs under Context-Free Grammars](https://mlsanity.com/paper/2602.00612/) (2026)
- [Seeing the End at Step Zero: Accelerating Diffusion MLLMs via MLP Sparsity-Aware Truncation](https://mlsanity.com/paper/2607.14557/) (2026)
- [Improving Variable-Length Generation in Diffusion Language Models via Length Regularization](https://mlsanity.com/paper/2602.07546/) (2026)
- [Beyond Fully Random Masking: Attention-Guided Denoising and Optimization for Diffusion Language Models](https://mlsanity.com/paper/2606.12273/) (2026)
- [Factorization-Error-Free Discrete Diffusion Language Model via Speculative Decoding](https://mlsanity.com/paper/2605.14305/) (2026)
- [Vision Is Not Overhead: One-Pass Block Drafting for Lossless Speculative Decoding in Vision-Language Models](https://mlsanity.com/paper/2609.00355/) (2026)
- [DEdit: Iterative Draft Editing for Speculative Decoding](https://mlsanity.com/paper/2609.38510/) (2026)

## Abstract (arXiv)

Efficiency, as a critical practical challenge for LLM-driven agentic and reasoning systems, is increasingly constrained by the inherent latency of autoregressive (AR) decoding. Speculative decoding mitigates this cost through a draft-verify scheme, yet existing approaches rely on AR draft models (a.k.a., drafters), which introduce two fundamental issues: (1) step-wise uncertainty accumulation leads to a progressive collapse of trust between the target model and the drafter, and (2) inherently sequential decoding of AR drafters. Together, these factors cause limited speedups. In this paper, we show that a diffusion large language model (dLLM) drafters can naturally overcome these issues through its fundamentally different probabilistic modeling and efficient parallel decoding strategy. Building on this insight, we introduce DEER, an efficient speculative decoding framework that drafts with diffusion and verifies with AR models. To enable high-quality drafting, DEER employs a two-stage training pipeline to align the dLLM-based drafters with the target AR model, and further adopts single-step decoding to generate long draft segments. Experiments show DEER reaches draft acceptance lengths of up to 32 tokens, far surpassing the 10 tokens achieved by EAGLE-3. Moreover, on HumanEval with Qwen3-30B-A3B, DEER attains a 5.54x speedup, while EAGLE-3 achieves only 2.41x. Code, model, demo, etc, will be available at https://czc726.github.io/DEER/