# DUEL: Exact Likelihood for Masked Diffusion via Deterministic Unmasking arXiv: https://arxiv.org/abs/2603.01367 ยท HTML: https://mlsanity.com/paper/2603.01367/ ## Key facts - Cited by 9 papers in the map (data through 14 Oct 2026). - Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. - Builds on 58 of its 68 references that are in the map. - First posted on arXiv 2 Mar 2026. ## Placed in - Diffusion models / [Discrete and text diffusion](https://mlsanity.com/diffusion/discrete-and-text-diffusion/) ## Builds on (most cited first, 30 of 58) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023) - [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024) - [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020) - [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021) - [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015) - [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019) - [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016) - [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019) - [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015) - [Structured Denoising Diffusion Models in Discrete State-Spaces](https://mlsanity.com/paper/2107.03006/) (2021) - [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021) - [MaskGIT: Masked Generative Image Transformer](https://mlsanity.com/paper/2202.04200/) (2022) - [Large Language Models are not Fair Evaluators](https://mlsanity.com/paper/2305.17926/) (2023) - [Diffusion-LM Improves Controllable Text Generation](https://mlsanity.com/paper/2205.14217/) (2022) - [The LAMBADA dataset: Word prediction requiring a broad discourse context](https://mlsanity.com/paper/1606.06031/) (2016) - [Large Language Diffusion Models](https://mlsanity.com/paper/2502.09992/) (2025) - [Simple and Effective Masked Diffusion Language Models](https://mlsanity.com/paper/2406.07524/) (2024) - [Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution](https://mlsanity.com/paper/2310.16834/) (2023) - [Simplified and Generalized Masked Diffusion for Discrete Data](https://mlsanity.com/paper/2406.04329/) (2024) - [Argmax Flows and Multinomial Diffusion: Learning Categorical Distributions](https://mlsanity.com/paper/2102.05379/) (2021) - [A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents](https://mlsanity.com/paper/1804.05685/) (2018) - [MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers](https://mlsanity.com/paper/2102.01454/) (2021) - [A Continuous Time Framework for Discrete Denoising Models](https://mlsanity.com/paper/2205.14987/) (2022) - [Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding](https://mlsanity.com/paper/2505.22618/) (2025) - [Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data](https://mlsanity.com/paper/2406.03736/) (2024) - [Discrete Flow Matching](https://mlsanity.com/paper/2407.15595/) (2024) - [One Billion Word Benchmark for Measuring Progress in Statistical Language Modeling](https://mlsanity.com/paper/1312.3005/) (2013) - [Mercury: Ultra-Fast Language Models Based on Diffusion](https://mlsanity.com/paper/2506.17298/) (2025) - [Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs](https://mlsanity.com/paper/2402.03927/) (2024) ## Built on by (most cited first, 9 of 9) - [GDSD: Reinforcement Learning as Guided Denoiser Self-Distillation for Diffusion Language Models](https://mlsanity.com/paper/2605.29398/) (2026) - [A Tale of Two Temperatures: Simple, Efficient, and Diverse Sampling from Diffusion Language Models](https://mlsanity.com/paper/2604.09921/) (2026) - [TUBE: Tangent Upper Bound on Evidence for Discrete Diffusion Language Models](https://mlsanity.com/paper/2605.24292/) (2026) - [Set Diffusion: Interpolating Token Orderings Between Autoregression and Diffusion for Fast and Flexible Decoding](https://mlsanity.com/paper/2607.01775/) (2026) - [Adaptive Order Policies for Masked Diffusion](https://mlsanity.com/paper/2606.00295/) (2026) - [Simple Diffusion Language Models Are More Effective Few-Step Generators Than Reported](https://mlsanity.com/paper/2609.33947/) (2026) - [Fork-dLLM: Avoiding the Flexibility Trap in Diffusion Language Models](https://mlsanity.com/paper/2609.39859/) (2026) - [Language Models as AI Research World Models](https://mlsanity.com/paper/2610.12235/) (2026) - [Revisiting the 16x Pretraining Compute Gap between Diffusion and Autoregressive Language Models](https://mlsanity.com/paper/2610.17590/) (2026) ## Abstract (arXiv) Masked diffusion models (MDMs) generate text by iteratively selecting positions to unmask and then predicting tokens at those positions. Yet MDMs lack proper likelihood evaluation: the evidence lower bound (ELBO) is not only a loose bound on log-likelihood, but, as we show, is also computed under the training distribution rather than the test-time distribution. We resolve this within our DUEL framework, which unifies leading MDM sampling strategies that employ textitdeterministic position selection. We prove that DUEL samplers admit exact likelihood computation under the test-time distribution -- giving MDMs textitproper likelihood, and hence proper perplexity, for the first time. This proper perplexity is the natural analogue of autoregressive perplexity and lets us revisit key questions about MDMs. MDMs are substantially better than previously thought: the MDM-autoregressive perplexity gap shrinks by up to 32% on in-domain data and 82% on zero-shot benchmarks. DUEL enables the first principled comparison of fast,parallel samplers across compute budgets -- an analysis impossible with the ELBO and unreliable with generative perplexity -- identifying a strong default method. Finally, oracle search over position orderings reveals MDMs can far surpass autoregressive models -- achieving 36.47 vs. 52.11 perplexity on AG News -- demonstrating the ceiling of MDM performance has not yet been reached.