mlsanityData to 7 Oct 2026

Diffusion Language Models Know the Answer Before Decoding

arXiv 2508.19982 ↗First posted 27 Aug 2025cs.CL cs.AIcited by 68

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Diffusion language models (DLMs) have recently emerged as an alternative to autoregressive approaches, offering parallel sequence generation and flexible token orders. However, their inference remains slower than that of autoregressive models, primarily due to the cost of bidirectional attention and the large number of refinement steps required for high quality outputs. In this work, we highlight and leverage an overlooked property of DLMs early answer convergence: in many cases, the correct answer can be internally identified by half steps before the final decoding step, both under semi-autoregressive and random remasking schedules. For example, on GSM8K and MMLU, up to 97% and 99% of instances, respectively, can be decoded correctly using only half of the refinement steps. Building on this observation, we introduce Prophet, a training-free fast decoding paradigm that enables early commit decoding. Specifically, Prophet dynamically decides whether to continue refinement or to go "all-in" (i.e., decode all remaining tokens in one step), using the confidence gap between the top-2 prediction candidates as the criterion. It integrates seamlessly into existing DLM implementations, incurs negligible overhead, and requires no additional training. Empirical evaluations of LLaDA-8B and Dream-7B across multiple tasks show that Prophet reduces the number of decoding steps by up to 3.4x while preserving high generation quality. These results recast DLM decoding as a problem of when to stop sampling, and demonstrate that early decode convergence provides a simple yet powerful mechanism for accelerating DLM inference, complementary to existing speedup techniques. Our code is publicly available at https://github.com/pixeli99/Prophet.

Key facts. Cited by 68 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 41 of its 50 references that are in the map. First posted on arXiv 27 Aug 2025.

Builds on

41 of 50 references.

Closest ancestors · 16

Same direction, newest first

  1. 2025Dream 7B: Diffusion Large Language Models21 Aug
  2. 2025DPad: Efficient Diffusion Language Models with Suffix Dropout19 Aug
  3. 2025Time Is a Feature: Exploiting Temporal Dynamics in Diffusion Language Models12 Aug
  4. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing8 Aug
  5. 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference4 Aug
  6. 2025FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion27 May
  7. 2025dKV-Cache: The Cache for Diffusion Language Models21 May
  8. 2025CtrlDiff: Boosting Large Diffusion Language Models with Dynamic Block Prediction and Controllable Generation20 May
  9. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
  10. 2025Large Language Diffusion Models14 Feb
  11. 2024Beyond Autoregression: Fast LLMs via Self-Distillation Through Time28 Oct
  12. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models23 Oct
4 more
  1. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling4 Sep
  2. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  3. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  4. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun

Other references · 25

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  6. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  8. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  9. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  10. 2021Program Synthesis with Large Language Modelscited by 3,509
  11. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  12. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
13 more
  1. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  2. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  3. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  4. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  5. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  6. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
  7. 2022Torsional Diffusion for Molecular Conformer Generationcited by 262
  8. 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 225
  9. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 168
  10. 2025Accelerating Diffusion LLMs via Adaptive Parallel Decodingcited by 108
  11. 2025Sparse-dLLM: Accelerating Diffusion LLMs with Dynamic Cache Evictioncited by 64
  12. 2025Judge Decoding: Faster Speculative Sampling Requires Going Beyond Model Alignmentcited by 59
  13. 2025Accelerating Diffusion Large Language Models with SlowFast Sampling: The Three Golden Principlescited by 43

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 22; 2026: 46.

68 papers.

Sort:

By area:

  1. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 128
  2. 2025dParallel: Learnable Parallel Decoding for dLLMscited by 59
  3. 2025KLASS: KL-Guided Fast Inference in Masked Diffusion Modelscited by 44
  4. 2025AdaBlock-dLLM: Semantic-Aware Diffusion LLM Inference via Adaptive Block Sizecited by 30
  5. 2025Self Speculative Decoding for Diffusion Large Language Modelscited by 29
  6. 2025Learning Unmasking Policies for Diffusion Language Modelscited by 29
  7. 2025Learning to Parallel: Accelerating Diffusion Large Language Models via Learnable Parallel Decodingcited by 26
  8. 2025d2Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Cachingcited by 24
  9. 2025Accelerating Diffusion LLM Inference via Local Determinism Propagationcited by 21
  10. 2026Flow caching for autoregressive video generationcited by 19
  11. 2025LoPA: Scaling dLLM Inference via Lookahead Parallel Decodingcited by 17
  12. 2025dVLM-AD: Enhance Diffusion Vision-Language-Model for Driving via Controllable Reasoningcited by 14
38 more
  1. 2026Deferred Commitment Decoding for Diffusion Language Modelscited by 11
  2. 2026Why Diffusion Language Models Struggle with Truly Parallel (Non-Autoregressive) Decoding?cited by 11
  3. 2025Fast-Decoding Diffusion Language Models via Progress-Aware Confidence Schedulescited by 10
  4. 2025Advancing Reasoning in Diffusion Language Models with Denoising Process Rewardscited by 9
  5. 2025Latent Refinement Decoding: Enhancing Diffusion-Based Language Models by Refining Belief Statescited by 9
  6. 2025Beyond Confidence: Adaptive and Coherent Decoding for Diffusion Language Modelscited by 9
  7. 2025Planned Diffusioncited by 8
  8. 2025Improving the Throughput of Diffusion-based Large Language Models via a Training-Free Confidence-Aware Calibrationcited by 7
  9. 2026DLLM-Searcher: Adapting Diffusion Large Language Model for Search Agentscited by 7
  10. 2025Orchestrating Dual-Boundaries: An Arithmetic Intensity Inspired Acceleration Framework for Diffusion Language Modelscited by 5
  11. 2026Early Decisions Matter: Proximity Bias and Initial Trajectory Shaping in Non-Autoregressive Diffusion Language Modelscited by 4
  12. 2025Planner and Executor: Collaboration between Discrete Diffusion And Autoregressive Models in Reasoningcited by 3
  13. 2025Rethinking Reasoning with MDLMs: Early Exits, Post-hoc Reasoning, and Beyondcited by 3
  14. 2025A Comprehensive Study on Visual Token Redundancy for Discrete Diffusion-based Multimodal Large Language Modelscited by 3
  15. 2025Diffusion Language Model Inference with Monte Carlo Tree Searchcited by 3
  16. 2026Search or Accelerate: Confidence-Switched Position Beam Search for Diffusion Language Modelscited by 3
  17. 2026Breaking Block Boundaries: Anchor-based History-stable Decoding for Diffusion Large Language Modelscited by 3
  18. 2026Unlocking the Potentials of Retrieval-Augmented Generation for Diffusion Language Modelscited by 2
  19. 2026TIDE: Efficient and Lossless MoE Diffusion LLM Inference with I/O-aware Expert Offloadcited by 2
  20. 2026TACG: Trajectory-Aware Commit Gating for Diffusion Language Model Decodingcited by 2
  21. 2026DIP: Dynamic In-Context Planner For Diffusion Language Modelscited by 1
  22. 2026Just on Time: Token-Level Early Stopping for Diffusion Language Modelscited by 1
  23. 2026One-Token Verification for Reasoning Correctness Estimationcited by 1
  24. 2026LADR: Locality-Aware Dynamic Rescue for Efficient Text-to-Image Generation with Diffusion Large Language Modelscited by 1
  25. 2026GeoBlock: Inferring Block Granularity from Dependency Geometry in Diffusion Language Modelscited by 1
  26. 2026STDec: Spatio-Temporal Stability Guided Decoding for dLLMscited by 1
  27. 2026Re-Mask and Redirect: Exploiting Denoising Irreversibility in Diffusion Language Modelscited by 1
  28. 2026Differences in Text Generated by Diffusion and Autoregressive Language Modelscited by 1
  29. 2026The Path Matters: Learning a Token-Commitment Policy for Diffusion Language Modelscited by 1
  30. 2026Optimus: Elastic Decoding for Efficient Diffusion LLM Servingcited by 1
  31. 2026Efficient Diffusion LLMs via Temporal-Spatial Parallel Decoding and Confidence Extrapolationcited by 1
  32. 2026Follow the Latent Roadmap: Navigating Revocable Decoding for Diffusion LLMs with Anchor Tokenscited by 1
  33. 2026VoidPadding: Let [VOID] Handle Padding in Masked Diffusion Language Models so that [EOS] Can Focus on Semantic Terminationcited by 1
  34. 2026Where to Place the Query? Unveiling and Mitigating Positional Bias in In-Context Learning for Diffusion LLMs via Decoding Dynamicscited by 1
  35. 2026Scheduling Thoughts: Learning the Order of Thought in Diffusion Language Modelscited by 1
  36. 2026Polestar: Drift-Aware Cache Calibration and Token Commitment for Efficient Inference of Diffusion LLMscited by 1
  37. 2026Temporal Self-Distillation: Faster Inference in Discrete Diffusion Language Modelscited by 1
  38. 2026ELF-REG: Scaling Continuous Diffusion Language Models to Reasoning Taskscited by 1

50 most cited of 68 All 68 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Sequential Diffusion Language Models
  2. 2025From Bits to Rounds: Parallel Decoding with Exploration for Diffusion Language Models
  3. 2025Beyond Next-Token Prediction: A Performance Characterization of Diffusion versus Autoregressive Language Models
  4. 2026Locally Coherent Parallel Decoding in Diffusion Language Models
  5. 2025Finish First, Perfect Later: Test-Time Token-Level Cross-Validation for Diffusion Large Language Models
  6. 2026Confidence-Based Decoding is Provably Efficient for Diffusion Language Models
  7. 2026Introspective Diffusion Language Models
  8. 2025Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Models