mlsanityData to 14 Oct 2026

Speculative Diffusion Decoding: Accelerating Language Generation through Diffusion

arXiv 2408.05636 ↗First posted 10 Aug 2024cs.CL cs.LGcited by 57

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

Speculative decoding has emerged as a widely adopted method to accelerate large language model inference without sacrificing the quality of the model outputs. While this technique has facilitated notable speed improvements by enabling parallel sequence verification, its efficiency remains inherently limited by the reliance on incremental token generation in existing draft models. To overcome this limitation, this paper proposes an adaptation of speculative decoding which uses discrete diffusion models to generate draft sequences. This allows parallelization of both the drafting and verification steps, providing significant speedups to the inference process. Our proposed approach, textitSpeculative Diffusion Decoding (SpecDiff), is validated on standard language generation benchmarks and empirically demonstrated to provide up to 7.2x speedups over standard generation processes and up to 1.75x speedups over existing speculative decoding approaches.

Key facts. Cited by 57 papers in the map (data through 14 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 29 of its 44 references that are in the map. First posted on arXiv 10 Aug 2024.

Builds on

29 of 44 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees24 Jun
  2. 2024SpecExec: Massively Parallel Speculative Decoding for Interactive LLM Inference on Consumer Devices4 Jun
  3. 2024Recurrent Drafter for Fast Speculative Decoding in Large Language Models14 Mar
  4. 2024Break the Sequential Dependency of LLM Inference Using Lookahead Decoding3 Feb
  5. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty26 Jan
  6. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads19 Jan
  7. 2024Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding15 Jan
  8. 2023Accelerating Large Language Model Decoding with Speculative Sampling2 Feb
  9. 2022Fast Inference from Transformers via Speculative Decoding30 Nov
  10. 2022Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generation30 Mar

Other references · 19

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  6. 2020Scaling Laws for Neural Language Modelscited by 6,586
  7. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  8. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  9. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  10. 2024Simple and Effective Masked Diffusion Language Modelscited by 848
  11. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 812
  12. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 745
7 more
  1. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 574
  2. 2024Better & Faster Large Language Models via Multi-token Predictioncited by 360
  3. 2018Blockwise Parallel Decoding for Deep Autoregressive Modelscited by 323
  4. 2024Rolling Diffusion Modelscited by 104
  5. 2023Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systemscited by 102
  6. 2022The Goldilocks of Pragmatic Understanding: Fine-Tuning Strategy Matters for Implicature Resolution by LLMscited by 58
  7. 2024Decoding Compressed Trust: Scrutinizing the Trustworthiness of Efficient LLMs Under Compressioncited by 57

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 24; 2026: 32.

57 papers.

Sort:

By area:

  1. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmaskingcited by 131
  2. 2025Accelerating Diffusion LLMs via Adaptive Parallel Decodingcited by 112
  3. 2025A Survey on Diffusion Language Modelscited by 78
  4. 2025FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusioncited by 77
  5. 2025Judge Decoding: Faster Speculative Sampling Requires Going Beyond Model Alignmentcited by 59
  6. 2025DiffuSpec: Unlocking Diffusion Language Models for Speculative Decodingcited by 42
  7. 2024Efficient Diffusion Models: A Comprehensive Survey from Principles to Practicescited by 41
  8. 2026Domino: Decoupling Causal Modeling from Autoregressive Drafting in Speculative Decodingcited by 34
  9. 2025Accelerated Diffusion Models via Speculative Samplingcited by 31
  10. 2025Self Speculative Decoding for Diffusion Large Language Modelscited by 30
  11. 2025SpecDiff-2: Scaling Diffusion Drafter Alignment For Faster Speculative Decodingcited by 26
  12. 2026DART: Diffusion-Inspired Speculative Decoding for Fast LLM Inferencecited by 24
38 more
  1. 2025Set Block Decoding is a Language Model Inference Acceleratorcited by 23
  2. 2025Structuring The Future: Diffusion LLM Speculative Decoding via Calibrated Draft Graphscited by 21
  3. 2025CDLM: Consistency Diffusion Language Models For Faster Samplingcited by 21
  4. 2025Playing with Transformer at 30+ FPS via Next-Frame Diffusioncited by 20
  5. 2025Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Modelscited by 19
  6. 2025Constrained Discrete Diffusioncited by 17
  7. 2025A Survey on Cache Methods in Diffusion Models: Toward Efficient Multi-Modal Generationcited by 16
  8. 2026Speculative Speculative Decodingcited by 15
  9. 2026dLLM: Simple Diffusion Language Modelingcited by 14
  10. 2025Beyond Next-Token Prediction: A Performance Characterization of Diffusion versus Autoregressive Language Modelscited by 13
  11. 2025Speeding up Speculative Decoding via Sequential Approximate Verificationcited by 11
  12. 2026Why Diffusion Language Models Struggle with Truly Parallel (Non-Autoregressive) Decoding?cited by 11
  13. 2025DEER: Draft with Diffusion, Verify with Autoregressive Modelscited by 9
  14. 2026DFlare: Scaling Up Draft Capacity for Block Diffusion Speculative Decodingcited by 9
  15. 2026CD4LM: Consistency Distillation and aDaptive Decoding for Diffusion Language Modelscited by 8
  16. 2026PARD-2: Target-Aligned Parallel Draft Model for Dual-Mode Speculative Decodingcited by 8
  17. 2025Speculative Jacobi-Denoising Decoding for Accelerating Autoregressive Text-to-image Generationcited by 7
  18. 2026DARTree: Speculative Diffusion Decoding with Autoregressive Draft Treescited by 6
  19. 2025Dimathtt[M]O: Distilling Masked Diffusion Models into One-step Generatorcited by 5
  20. 2026Verification-Aware Training for Speculative Decodingcited by 5
  21. 2026ECHO: Efficient Chest X-ray Report Generation with One-step Block Diffusioncited by 4
  22. 2026Simple Self-Conditioning Adaptation for Masked Diffusion Modelscited by 4
  23. 2026DeLS-Spec: Decoupled Long-Short Contexts for Parallel Speculative Draftingcited by 4
  24. 2025Efficient Perplexity Bound and Ratio Matching in Discrete Diffusion Language Modelscited by 3
  25. 2026TRIMS: Trajectory-Ranked Instruction Masked Supervision for Diffusion Language Modelscited by 2
  26. 2026FlexDraft: Flexible Speculative Decoding via Attention Tuning and Bonus-Guided Calibrationcited by 2
  27. 2026SimSD: Simple Speculative Decoding in Diffusion Language Modelscited by 2
  28. 2026Accelerating Speculative Diffusions via Block Verificationcited by 2
  29. 2026A2D2: Fine-Tuning Any-Length Discrete Diffusion for Adaptive Decodingcited by 2
  30. 2026Accelerating Masked Diffusion Large Language Models: A Survey of Efficient Inference Techniquescited by 1
  31. 2026Draft in Parallel, Condition Through Depth: Adjacent Causal Injection for Speculative Decodingcited by 1
  32. 2025Automatic Task Detection and Heterogeneous LLM Speculative Decodingcited by 0
  33. 2025READER: Retrieval-Assisted Drafter for Efficient LLM Inferencecited by 0
  34. 2026Progressive Refinement Regulation for Accelerating Diffusion Language Model Decodingcited by 0
  35. 2026Draft-and-Target Sampling for Video Generation Policycited by 0
  36. 2026Multi-Token Residual Predictioncited by 0
  37. 2026Diffusion Language Model Parallel Decoding via Product-of-Experts Bridgecited by 0
  38. 2026Speculative Refinement: A Hybrid Autoregressive Diffusion Decoding Strategy and Its Behavior Across Benchmarkscited by 0

50 most cited of 57 All 57 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026D^2SD: Accelerating Speculative Decoding with Dual Diffusion Draft Models
  2. 2026PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decoding
  3. 2026AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters
  4. 2026Trajectory-Level Speculative Decoding for Diffusion Language Models
  5. 2026DualDiffusion: A Speculative Decoding Strategy for Masked Diffusion Models
  6. 2025Fail Fast, Win Big: Rethinking the Drafting Strategy in Speculative Decoding via Diffusion LLMs
  7. 2024A Theoretical Perspective for Speculative Decoding Algorithm
  8. 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference