# DART: Diffusion-Inspired Speculative Decoding for Fast LLM Inference

arXiv: https://arxiv.org/abs/2601.19278 · HTML: https://mlsanity.com/paper/2601.19278/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving.
- Builds on 29 of its 39 references that are in the map.
- First posted on arXiv 27 Jan 2026.

## Placed in

- Efficient LLM inference and serving / [Speculative decoding](https://mlsanity.com/inference/speculative-decoding/)

## Builds on (most cited first, 29 of 29)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [Fast Inference from Transformers via Speculative Decoding](https://mlsanity.com/paper/2211.17192/) (2022)
- [Accelerating Large Language Model Decoding with Speculative Sampling](https://mlsanity.com/paper/2302.01318/) (2023)
- [Large Language Diffusion Models](https://mlsanity.com/paper/2502.09992/) (2025)
- [Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads](https://mlsanity.com/paper/2401.10774/) (2024)
- [Enhancing Chat Language Models by Scaling High-quality Instructional Conversations](https://mlsanity.com/paper/2305.14233/) (2023)
- [Dream 7B: Diffusion Large Language Models](https://mlsanity.com/paper/2508.15487/) (2025)
- [Datasets: A Community Library for Natural Language Processing](https://mlsanity.com/paper/2109.02846/) (2021)
- [EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees](https://mlsanity.com/paper/2406.16858/) (2024)
- [EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test](https://mlsanity.com/paper/2503.01840/) (2025)
- [Break the Sequential Dependency of LLM Inference Using Lookahead Decoding](https://mlsanity.com/paper/2402.02057/) (2024)
- [Mercury: Ultra-Fast Language Models Based on Diffusion](https://mlsanity.com/paper/2506.17298/) (2025)
- [Flex Attention: A Programming Model for Generating Optimized Attention Kernels](https://mlsanity.com/paper/2412.05496/) (2024)
- [SpecTr: Fast Speculative Decoding via Optimal Transport](https://mlsanity.com/paper/2310.15141/) (2023)
- [TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding](https://mlsanity.com/paper/2404.11912/) (2024)
- [Learning Harmonized Representations for Speculative Sampling](https://mlsanity.com/paper/2408.15766/) (2024)
- [Speculative Diffusion Decoding: Accelerating Language Generation through Diffusion](https://mlsanity.com/paper/2408.05636/) (2024)
- [DiffuSpec: Unlocking Diffusion Language Models for Speculative Decoding](https://mlsanity.com/paper/2510.02358/) (2025)
- [SwapMoE: Serving Off-the-shelf MoE-based Large Language Models with Tunable Memory Budget](https://mlsanity.com/paper/2308.15030/) (2023)
- [SpecDiff-2: Scaling Diffusion Drafter Alignment For Faster Speculative Decoding](https://mlsanity.com/paper/2511.00606/) (2025)
- [Falcon: Faster and Parallel Inference of Large Language Models through Enhanced Semi-Autoregressive Drafting and Custom-Designed Decoding Tree](https://mlsanity.com/paper/2412.12639/) (2024)
- [DEER: Draft with Diffusion, Verify with Autoregressive Models](https://mlsanity.com/paper/2512.15176/) (2025)
- [Memory Offloading for Large Language Model Inference with Latency SLO Guarantees](https://mlsanity.com/paper/2502.08182/) (2025)

## Built on by (most cited first, 24 of 24)

- [Domino: Decoupling Causal Modeling from Autoregressive Drafting in Speculative Decoding](https://mlsanity.com/paper/2605.29707/) (2026)
- [Accelerating Speculative Decoding with Block Diffusion Draft Trees](https://mlsanity.com/paper/2604.12989/) (2026)
- [D-PACE: Dynamic Position-Aware Cross-Entropy for Parallel Speculative Drafting](https://mlsanity.com/paper/2605.18810/) (2026)
- [Draft-OPD: On-Policy Distillation for Speculative Draft Models](https://mlsanity.com/paper/2605.29343/) (2026)
- [TAPS: Target-Aware Prefix Tree Selection for Diffusion-Drafted Speculative Decoding](https://mlsanity.com/paper/2606.00487/) (2026)
- [PARD-2: Target-Aligned Parallel Draft Model for Dual-Mode Speculative Decoding](https://mlsanity.com/paper/2605.08632/) (2026)
- [Cost-Aware Diffusion Draft Trees for Speculative Decoding](https://mlsanity.com/paper/2606.01813/) (2026)
- [Seeing the End at Step Zero: Accelerating Diffusion MLLMs via MLP Sparsity-Aware Truncation](https://mlsanity.com/paper/2607.14557/) (2026)
- [TreeFlash: Parallel AR-Approximation for Faster Speculative Decoding](https://mlsanity.com/paper/2606.03819/) (2026)
- [DeLS-Spec: Decoupled Long-Short Contexts for Parallel Speculative Drafting](https://mlsanity.com/paper/2607.07409/) (2026)
- [AngelSpec: Towards Real-World High Performance Inference with Speculative Decoding](https://mlsanity.com/paper/2607.25852/) (2026)
- [SpecBlock: Block-Iterative Speculative Decoding with Dynamic Tree Drafting](https://mlsanity.com/paper/2605.07243/) (2026)
- [FlexDraft: Flexible Speculative Decoding via Attention Tuning and Bonus-Guided Calibration](https://mlsanity.com/paper/2605.20022/) (2026)
- [DFlow: Enabling Verifier Information Flow in Block Diffusion Speculative Decoding](https://mlsanity.com/paper/2609.06498/) (2026)
- [When Parallel Drafter Meets Parallel Speculative Decoding](https://mlsanity.com/paper/2609.27396/) (2026)
- [CURE: Local Uncertainty Repair for Block-Parallel Speculative Decoding](https://mlsanity.com/paper/2608.00531/) (2026)
- [Draft in Parallel, Condition Through Depth: Adjacent Causal Injection for Speculative Decoding](https://mlsanity.com/paper/2609.36173/) (2026)
- [Enabling Performant and Flexible Model-Internal Observability for LLM Inference](https://mlsanity.com/paper/2605.11093/) (2026)
- [Fully Rotation-Equivariant Spectral-Spatial Learning for Multispectral Object Detection](https://mlsanity.com/paper/2607.05148/) (2026)
- [Accepted Prefixes Are Not All You Need: A Negative Result on PEFT-Based Block-Diffusion Drafting](https://mlsanity.com/paper/2607.12422/) (2026)
- [LibraSpec: Dynamic Diffusion-Based Speculative Decoding via Marginal-Gain-Driven Optimization](https://mlsanity.com/paper/2608.08721/) (2026)
- [UBTree: Parallel Tree Drafting via Unigram and Bigram Models for Speculative Decoding](https://mlsanity.com/paper/2609.39972/) (2026)
- [CAST: Cost-Aware Speculative Trees from One-Pass Block Drafters](https://mlsanity.com/paper/2610.00321/) (2026)
- [AdaSpark: Adaptive DSpark with Online Learning for Tree Verification and N-gram Fill](https://mlsanity.com/paper/2610.05774/) (2026)

## Abstract (arXiv)

Speculative decoding is an effective and lossless approach for accelerating LLM inference. However, existing widely adopted model-based draft designs, such as EAGLE3, improve accuracy at the cost of multi-step autoregressive inference, resulting in high drafting latency and ultimately rendering the drafting stage itself a performance bottleneck. Inspired by diffusion-based large language models (dLLMs), we propose DART, which leverages parallel generation to reduce drafting latency. DART predicts logits for multiple future masked positions in parallel within a single forward pass based on hidden states of the target model, thereby eliminating autoregressive rollouts in the draft model while preserving a lightweight design. Based on these parallel logit predictions, we further introduce an efficient tree pruning algorithm that constructs high-quality draft token trees with N-gram-enforced semantic continuity. DART substantially reduces draft-stage overhead while preserving high draft accuracy, leading to significantly improved end-to-end decoding speed. Experimental results demonstrate that DART achieves a 2.03x--3.44x wall-clock time speedup across multiple datasets, surpassing EAGLE3 by 30% on average and offering a practical speculative decoding framework. Code is released at https://github.com/fvliang/DART.