mlsanityData to 7 Oct 2026

DUEL: Exact Likelihood for Masked Diffusion via Deterministic Unmasking

arXiv 2603.01367 ↗First posted 2 Mar 2026cs.LGcited by 8

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Masked diffusion models (MDMs) generate text by iteratively selecting positions to unmask and then predicting tokens at those positions. Yet MDMs lack proper likelihood evaluation: the evidence lower bound (ELBO) is not only a loose bound on log-likelihood, but, as we show, is also computed under the training distribution rather than the test-time distribution. We resolve this within our DUEL framework, which unifies leading MDM sampling strategies that employ textitdeterministic position selection. We prove that DUEL samplers admit exact likelihood computation under the test-time distribution -- giving MDMs textitproper likelihood, and hence proper perplexity, for the first time. This proper perplexity is the natural analogue of autoregressive perplexity and lets us revisit key questions about MDMs. MDMs are substantially better than previously thought: the MDM-autoregressive perplexity gap shrinks by up to 32% on in-domain data and 82% on zero-shot benchmarks. DUEL enables the first principled comparison of fast,parallel samplers across compute budgets -- an analysis impossible with the ELBO and unreliable with generative perplexity -- identifying a strong default method. Finally, oracle search over position orderings reveals MDMs can far surpass autoregressive models -- achieving 36.47 vs. 52.11 perplexity on AG News -- demonstrating the ceiling of MDM performance has not yet been reached.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 58 of its 68 references that are in the map. First posted on arXiv 2 Mar 2026.

Builds on

58 of 68 references.

Closest ancestors · 24

Same direction, newest first

  1. 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100B10 Dec
  2. 2025Learning Unmasking Policies for Diffusion Language Models9 Dec
  3. 2025KLASS: KL-Guided Fast Inference in Masked Diffusion Models7 Nov
  4. 2025Encoder-Decoder Diffusion Language Models for Efficient Training and Inference26 Oct
  5. 2025Demystifying MaskGIT Sampler and Beyond: Adaptive Order Selection in Masked Diffusion6 Oct
  6. 2025Don't Settle Too Early: Self-Reflective Remasking for Diffusion Language Models28 Sep
  7. 2025Planner Aware Path Learning in Diffusion Language Models Training27 Sep
  8. 2025d2: Improving Reasoning in Diffusion Language Models via Trajectory Likelihood Estimation25 Sep
  9. 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference4 Aug
  10. 2025Plan for Speed: Dilated Scheduling for Masked Diffusion Language Models23 Jun
  11. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking30 May
  12. 2025Remasking Discrete Diffusion Models with Inference-Time Scaling1 Mar
12 more
  1. 2025Large Language Diffusion Models14 Feb
  2. 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions10 Feb
  3. 2025Path Planning for Masked Diffusion Model Sampling5 Feb
  4. 2024Beyond Autoregression: Fast LLMs via Self-Distillation Through Time28 Oct
  5. 2024Scaling up Masked Diffusion Models on Text24 Oct
  6. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling4 Sep
  7. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  8. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  9. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun
  10. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct
  11. 2023Likelihood-Based Diffusion Language Models30 May
  12. 2021Autoregressive Diffusion Models5 Oct

Other references · 34

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  7. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  8. 2016Pointer Sentinel Mixture Modelscited by 2,992
  9. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  10. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  11. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  12. 2021Variational Diffusion Modelscited by 1,145
22 more
  1. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  2. 2023Large Language Models are not Fair Evaluatorscited by 1,050
  3. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  4. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 960
  5. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  6. 2018A Discourse-Aware Attention Model for Abstractive Summarization of Long Documentscited by 477
  7. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 466
  8. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  9. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
  10. 2013One Billion Word Benchmark for Measuring Progress in Statistical Language Modelingcited by 302
  11. 2024Discrete Flow Matchingcited by 301
  12. 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 225
  13. 2024Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMscited by 224
  14. 2024Simple Guidance Mechanisms for Discrete Diffusion Modelscited by 148
  15. 2016Neural Autoregressive Distribution Estimationcited by 94
  16. 2013A Deep and Tractable Density Estimatorcited by 67
  17. 2025dParallel: Learnable Parallel Decoding for dLLMscited by 59
  18. 2023InfoDiffusion: Representation Learning Using Information Maximizing Diffusion Modelscited by 48
  19. 2024Distillation of Discrete Diffusion through Dimensional Correlationscited by 46
  20. 2025Learning-Order Autoregressive Models with Application to Molecular Graph Generationcited by 29
  21. 2025Reviving Any-Subset Autoregressive Models with Principled Parallel Sampling and Speculative Decodingcited by 14
  22. 2025Information-Theoretic Discrete Diffusioncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026A Tale of Two Temperatures: Simple, Efficient, and Diverse Sampling from Diffusion Language Modelscited by 5
  2. 2026GDSD: Reinforcement Learning as Guided Denoiser Self-Distillation for Diffusion Language Modelscited by 5
  3. 2026Set Diffusion: Interpolating Token Orderings Between Autoregression and Diffusion for Fast and Flexible Decodingcited by 2
  4. 2026TUBE: Tangent Upper Bound on Evidence for Discrete Diffusion Language Modelscited by 1
  5. 2026Adaptive Order Policies for Masked Diffusioncited by 0
  6. 2026Simple Diffusion Language Models Are More Effective Few-Step Generators Than Reportedcited by 0
  7. 2026Fork-dLLM: Avoiding the Flexibility Trap in Diffusion Language Modelscited by 0
  8. 2026Language Models as AI Research World Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Masking
  2. 2025Parallel Sampling from Masked Diffusion Models via Conditional Independence Testing
  3. 2025Error Bounds and Optimal Schedules for Masked Diffusions with Factorized Approximations
  4. 2025Masked Diffusion Models are Secretly Learned-Order Autoregressive Models
  5. 2025Optimal Inference Schedules for Masked Diffusion Models
  6. 2025Bringing Stability to Diffusion: Decomposing and Reducing Variance of Training Masked Diffusion Models
  7. 2026Learn from Your Mistakes: Self-Correcting Masked Diffusion Models
  8. 2025Any-Order Flexible Length Masked Diffusion