mlsanityData to 7 Oct 2026

Confidence-Based Decoding is Provably Efficient for Diffusion Language Models

arXiv 2603.22248 ↗First posted 23 Mar 2026cs.LG cs.AI cs.ITcited by 17

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Diffusion language models (DLMs) have emerged as a promising alternative to autoregressive (AR) models for language modeling, allowing flexible generation order and parallel generation of multiple tokens. However, this flexibility introduces a challenge absent in AR models: the decoding strategy -- which determines the order and number of tokens generated at each iteration -- critically affects sampling efficiency. Among decoding strategies explored in practice, confidence-based methods, which adaptively select which and how many tokens to unmask based on prediction confidence, have shown strong empirical performance. Despite this success, our theoretical understanding of confidence-based decoding remains limited. In this work, we develop the first theoretical analysis framework for confidence-based decoding in DLMs. We focus on an entropy sum-based strategy that continues unmasking tokens within each iteration until the cumulative entropy exceeds a threshold, and show that it achieves varepsilon-accurate sampling in KL divergence with an expected number of iterations widetilde O(H(X0)/varepsilon), where H(X0) denotes the entropy of the target data distribution. Notably, this strategy yields substantial sampling acceleration when the data distribution has low entropy relative to the sequence length, while automatically adapting to the intrinsic complexity of data without requiring prior knowledge or hyperparameter tuning. Overall, our results provide a theoretical foundation for confidence-based decoding and may inform the design of more efficient decoding strategies for DLMs.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 70 of its 76 references that are in the map. First posted on arXiv 23 Mar 2026.

Builds on

70 of 76 references.

Closest ancestors · 25

Same direction, newest first

  1. 2026Adaptation to Intrinsic Dependence in Diffusion Language Models23 Feb
  2. 2026Plan, Verify and Fill: A Structured Parallel Decoding Approach for Diffusion Language Models18 Jan
  3. 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100B10 Dec
  4. 2025From Bits to Rounds: Parallel Decoding with Exploration for Diffusion Language Models26 Nov
  5. 2025Optimal Inference Schedules for Masked Diffusion Models6 Nov
  6. 2025Lookahead Unmasking Elicits Accurate Decoding in Diffusion Language Models4 Nov
  7. 2025Error Bounds and Optimal Schedules for Masked Diffusions with Factorized Approximations29 Oct
  8. 2025Dream 7B: Diffusion Large Language Models21 Aug
  9. 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference4 Aug
  10. 2025Diffusion Beats Autoregressive in Data-Constrained Settings21 Jul
  11. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking30 May
  12. 2025Breaking AR's Sampling Bottleneck: Provable Acceleration via Diffusion Language Models27 May
13 more
  1. 2025Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decoding22 May
  2. 2025Remasking Discrete Diffusion Models with Inference-Time Scaling1 Mar
  3. 2025Large Language Diffusion Models14 Feb
  4. 2025Theoretical Benefit and Limitation of Diffusion Language Model13 Feb
  5. 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions10 Feb
  6. 2025Path Planning for Masked Diffusion Model Sampling5 Feb
  7. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models23 Oct
  8. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling4 Sep
  9. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  10. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  11. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun
  12. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct
  13. 2023A Reparameterized Discrete Diffusion Model for Text Generation11 Feb

Other references · 45

  1. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  2. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  3. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  4. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptionscited by 440
  5. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  6. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
  7. 2023Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localizationcited by 230
  8. 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 225
  9. 2022Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptionscited by 215
  10. 2022Convergence of score-based generative modeling for general data distributionscited by 179
  11. 2022Convergence for score-based generative modeling with polynomial complexitycited by 176
  12. 2023Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Datacited by 171
33 more
  1. 2023Diffusion Models are Minimax Optimal Distribution Estimatorscited by 159
  2. 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 145
  3. 2024Accelerating Convergence of Score-Based Diffusion Models, Provablycited by 74
  4. 2024A Sharp Convergence Theory for The Probability Flow ODEs of Diffusion Modelscited by 64
  5. 2024Optimal score estimation via empirical Bayes smoothingcited by 60
  6. 2024Convergence of Diffusion Models Under the Manifold Hypothesis in High-Dimensionscited by 60
  7. 2024Convergence Analysis of Probability Flow ODE for Score-based Generative Modelscited by 56
  8. 2024Adapting to Unknown Low-Dimensional Structures in Score-Based Diffusion Modelscited by 56
  9. 2024Breaking the Curse of Dimensionality: Diffusion Models Efficiently Learn Low-Dimensional Distributionscited by 53
  10. 2024O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptionscited by 48
  11. 2024How Discrete and Continuous Diffusion Meet: Comprehensive Analysis of Discrete Diffusion Models via a Stochastic Integral Frameworkcited by 48
  12. 2025Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithmscited by 47
  13. 2024Minimax Optimality of Score-based Diffusion Models: Beyond the Density Lower Bound Assumptionscited by 45
  14. 2024Convergence Analysis of Discrete Diffusion Model: Exact Implementation through Uniformizationcited by 44
  15. 2024Denoising diffusion probabilistic models are optimally adaptive to unknown low dimensionalitycited by 44
  16. 2024Learning general Gaussian mixtures with efficient score matchingcited by 43
  17. 2025Accelerating Diffusion Large Language Models with SlowFast Sampling: The Three Golden Principlescited by 43
  18. 2024Linear Convergence of Diffusion Models Under the Manifold Hypothesiscited by 42
  19. 2024Learning Mixtures of Gaussians Using Diffusion Modelscited by 41
  20. 2025Low-dimensional adaptation of diffusion models: Convergence in total variationcited by 38
  21. 2024Provable Acceleration for Diffusion Models under Minimal Assumptionscited by 27
  22. 2025Minimax Optimality of the Probability Flow ODE for Diffusion Modelscited by 23
  23. 2025Dimension-Free Convergence of Diffusion Models for Approximate Gaussian Mixturescited by 23
  24. 2025Absorb and Converge: Provable Convergence Guarantee for Absorbing Discrete Diffusion Modelscited by 20
  25. 2025Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Modelscited by 19
  26. 2026Efficient Sampling with Discrete Diffusion Models: Sharp and Adaptive Guaranteescited by 19
  27. 2025Generalization error bound for denoising score matching under relaxed manifold assumptioncited by 18
  28. 2025Faster Diffusion Models via Higher-Order Approximationcited by 16
  29. 2025Optimal Convergence Analysis of DDPM for General Distributionscited by 16
  30. 2025Diffusion Factor Models: Generating High-Dimensional Returns with Factor Structurecited by 12
  31. 2026Sharp Convergence Rates for Masked Diffusion Modelscited by 8
  32. 2025Sublinear iterations can suffice even for DDPMscited by 7
  33. 2026High-accuracy and dimension-free sampling with diffusionscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 17.

17 papers.

Sort:

By area:

  1. 2026Discrete-WAM: Unified Discrete Vision-Action Token Editing for World-Policy Learningcited by 3
  2. 2026Provably adaptive sampling with uniform and remasking discrete diffusion modelscited by 3
  3. 2026Diffusion Models Are Statistically Optimal for Learning Low-Dimensional Multi-Modal Distributionscited by 2
  4. 2026Neither Parallel Nor Sequential: How DiffusionGemma Actually Commits Tokenscited by 2
  5. 2026ConvergeFlow: Language Flow with Provable Convergence to Token Embeddingscited by 2
  6. 2026DPRM: A Plug-in Doob h transform-induced Token-Ordering Module for Discrete Diffusion Modelscited by 1
  7. 2026The Confidence Shortcut: A Reasoning Failure Mode of Masked Diffusion Modelscited by 1
  8. 2026Diffusion Models Adapt to Low-Dimensional Structure Under Flexible Coefficient Choicescited by 1
  9. 2026Conditional Total Correlation and the Serial Depth of Adaptive Parallel Samplingcited by 1
  10. 2026Focus on the Core: Empowering Diffusion Large Language Models by Self-Contrastcited by 0
  11. 2026Escaping Confidence Trap: Evolutionary Decoding for Mathematical Reasoning in Diffusion LLMscited by 0
  12. 2026Schedule optimization for tau-leaping in masked discrete diffusioncited by 0
5 more
  1. 2026Entropy Can Flow, or It Can Guide. Be Entropy. LEDFlow: Introducing Entropy-guided Generation Order into Uniform Discrete Flowcited by 0
  2. 2026Faster Block-Diffusion Serving with Distribution-Free Risk Guaranteescited by 0
  3. 2026HyperZip: Efficient Data Compression through Personalized Diffusion LLMs with Hypernetworkscited by 0
  4. 2026Training Parallel Speculative Draft Models by Directly Minimizing Expected Decoding Roundscited by 0
  5. 2026The Lattice of Transition Lawscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Beyond Confidence: Adaptive and Coherent Decoding for Diffusion Language Models
  2. 2026Re-evaluating Confidence Remasking in Masked Diffusion Language Models
  3. 2026LESS Is More: Mutual-Stability Sampling for Diffusion Language Models
  4. 2025Diffusion Language Models Know the Answer Before Decoding
  5. 2026Fork-dLLM: Avoiding the Flexibility Trap in Diffusion Language Models
  6. 2026Locally Confident, Globally Stuck: The Quality-Exploration Dilemma in Diffusion Language Models
  7. 2025Beyond Static Cutoffs: One-Shot Dynamic Thresholding for Diffusion Language Models
  8. 2026Early-Bird Decoding: Accelerating Diffusion LLMs with Learnable Block Sizes and Parallel Sampling