mlsanityData to 14 Oct 2026

Optimizing Decoding Paths in Masked Diffusion Models by Quantifying Uncertainty

arXiv 2512.21336 ↗First posted 24 Dec 2025cs.CL cs.AI cs.LGcited by 6

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Masked Diffusion Models (MDMs) offer flexible, non-autoregressive generation, but this freedom introduces a challenge: final output quality is highly sensitive to the decoding order. We are the first to formalize this issue, attributing the variability in output quality to the cumulative predictive uncertainty along a generative path. To quantify this uncertainty, we introduce Denoising Entropy, a computable metric that serves as an internal signal for evaluating generative process. Leveraging this metric, we propose two algorithms designed to optimize the decoding path: a post-hoc selection method and a real-time guidance strategy. Experiments demonstrate that our entropy-guided methods significantly improve generation quality, consistently boosting accuracy on challenging reasoning, planning, and code benchmarks. Our work establishes Denoising Entropy as a principled tool for understanding and controlling generation, effectively turning the uncertainty in MDMs from a liability into a key advantage for discovering high-quality solutions.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 42 of its 53 references that are in the map. First posted on arXiv 24 Dec 2025.

Builds on

42 of 53 references.

Closest ancestors · 20

Same direction, newest first

  1. 2025Dream 7B: Diffusion Large Language Models21 Aug
  2. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation25 Jun
  3. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking30 May
  4. 2025Reinforcing the Diffusion Chain of Lateral Thought with Diffusion Language Models15 May
  5. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
  6. 2025Remasking Discrete Diffusion Models with Inference-Time Scaling1 Mar
  7. 2025Large Language Diffusion Models14 Feb
  8. 2025Theoretical Benefit and Limitation of Diffusion Language Model13 Feb
  9. 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions10 Feb
  10. 2025Path Planning for Masked Diffusion Model Sampling5 Feb
  11. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models23 Oct
  12. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning18 Oct
8 more
  1. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling4 Sep
  2. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  3. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  4. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun
  5. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct
  6. 2023A Reparameterized Discrete Diffusion Model for Text Generation11 Feb
  7. 2022DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models28 Nov
  8. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Control31 Oct

Other references · 22

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2020Denoising Diffusion Implicit Modelscited by 9,781
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  7. 2023Let's Verify Step by Stepcited by 3,789
  8. 2021Program Synthesis with Large Language Modelscited by 3,637
  9. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  10. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  11. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  12. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
10 more
  1. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 819
  2. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 420
  3. 2024Discrete Flow Matchingcited by 317
  4. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 285
  5. 2025A General Framework for Inference-time Scaling and Steering of Diffusion Modelscited by 224
  6. 2020Understanding Neural Abstractive Summarization Models via Uncertaintycited by 34
  7. 2024The Entropy Enigma: Success and Failure of Entropy Minimizationcited by 24
  8. 2024Improving Open-Ended Text Generation via Adaptive Decodingcited by 21
  9. 2024Transformers Can Navigate Mazes With Multi-Step Predictioncited by 11
  10. 2023The Stable Entropy Hypothesis and Entropy-Aware Decoding: An Analysis and Algorithm for Robust Natural Language Generationcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026Neither Parallel Nor Sequential: How DiffusionGemma Actually Commits Tokenscited by 3
  2. 2026On the Trainability of Masked Diffusion Language Models via Blockwise Localitycited by 2
  3. 2026Lookahead Path Likelihood Optimization for Diffusion LLMscited by 1
  4. 2026D5P4: Partition Determinantal Point Process for Diversity in Parallel Discrete Diffusion Decodingcited by 1
  5. 2026Visual-Redundancy-Controlled Parallel Decoding for Diffusion-Based Multimodal Large Language Modelscited by 1
  6. 2026Decoding in Order-Agnostic Language Models: Chain-Rule Deviation and Uniform Spreadingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Generation Order and Parallel Decoding in Masked Diffusion Models: An Information-Theoretic Perspective
  2. 2025Masked Diffusion Models are Secretly Learned-Order Autoregressive Models
  3. 2026Not All Denoising Steps Are Equal: Model Scheduling for Faster Masked Diffusion Language Models
  4. 2026Speculative Refinement: A Hybrid Autoregressive Diffusion Decoding Strategy and Its Behavior Across Benchmarks
  5. 2026Improving Sampling for Masked Diffusion Models via Information Gain
  6. 2025Fewer Denoising Steps or Cheaper Per-Step Inference: Towards Compute-Optimal Diffusion Model Deployment
  7. 2026Path-Dependent Denoising: A Non-Conservative Field Perspective on Order Collapse in Diffusion Language Models
  8. 2025EVODiff: Entropy-aware Variance Optimized Diffusion Inference