mlsanityData to 14 Oct 2026

Improving Discrete Diffusion Unmasking Policies Beyond Explicit Reference Policies

arXiv 2510.05725 ↗First posted 7 Oct 2025cs.LG cs.AI cs.CLcited by 22

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Masked diffusion models (MDMs) have recently emerged as a novel framework for language modeling. MDMs generate sentences by iteratively denoising masked sequences, filling in [MASK] tokens step by step. Although MDMs support any-order sampling, performance is highly sensitive to the choice of which position to unmask next. Prior work typically relies on rule-based schedules (e.g., max-confidence, max-margin), which provide ad hoc improvements. In contrast, we replace these heuristics with a learned scheduler. Specifically, we cast denoising as a KL-regularized Markov decision process (MDP) with an explicit reference policy and optimize a regularized objective that admits policy improvement and convergence guarantees under standard assumptions. We prove that the optimized policy under this framework generates samples that more closely match the data distribution than heuristic schedules. Empirically, across four benchmarks, our learned policy consistently outperforms max-confidence: for example, on SUDOKU, where unmasking order is critical, it yields a 20.1% gain over random and a 11.2% gain over max-confidence. Code is available at https://github.com/chunsanHong/UPO.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 30 of its 37 references that are in the map. First posted on arXiv 7 Oct 2025.

Builds on

30 of 37 references.

Closest ancestors · 10

Same direction, newest first

  1. 2025Dream 7B: Diffusion Large Language Models21 Aug
  2. 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference4 Aug
  3. 2025Reinforcing the Diffusion Chain of Lateral Thought with Diffusion Language Models15 May
  4. 2025Large Language Diffusion Models14 Feb
  5. 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions10 Feb
  6. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling4 Sep
  7. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  8. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  9. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun
  10. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct

Other references · 20

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  7. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
  8. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  9. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  10. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  11. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 574
  12. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 451
8 more
  1. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 285
  2. 2025d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learningcited by 186
  3. 2025Reinforcement Learning with Verifiable Rewards: GRPO's Effective Loss, Dynamics, and Success Amplificationcited by 110
  4. 2022Training and Inference on Any-Order Autoregressive Models the Right Waycited by 64
  5. 2024Causal Language Modeling Can Elicit Search and Reasoning Capabilities on Logic Puzzlescited by 38
  6. 2025Fine-Tuning Discrete Diffusion Models with Policy Gradient Methodscited by 38
  7. 2023PPO-Clip Attains Global Optimality: Towards Deeper Understandings of Clippingcited by 17
  8. 2021Discovering Non-monotonic Autoregressive Orderings with Variational Inferencecited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 21.

22 papers.

Sort:

By area:

  1. 2025Learning Unmasking Policies for Diffusion Language Modelscited by 30
  2. 2026Stop Training for the Worst: Progressive Unmasking Accelerates Masked Diffusion Trainingcited by 10
  3. 2026Where-to-Unmask: Ground-Truth-Guided Unmasking Order Learning for Masked Diffusion Language Modelscited by 9
  4. 2026Unifying Masked Diffusion Models with Various Generation Orders and Beyondcited by 6
  5. 2026Locally Confident, Globally Stuck: The Quality-Exploration Dilemma in Diffusion Language Modelscited by 5
  6. 2026Fixed-Point Masked Generative Modelingcited by 5
  7. 2026Reinforcement Learning for Diffusion LLMs with Entropy-Guided Step Selection and Stepwise Advantagescited by 4
  8. 2026Attention-Discounted Adaptive Sampler for Masked Diffusion Language Modelscited by 2
  9. 2026Tuning the Implicit Regularizer of Masked Diffusion Language Models: Enhancing Generalization via Insights from k-Paritycited by 1
  10. 2026The Confidence Shortcut: A Reasoning Failure Mode of Masked Diffusion Modelscited by 1
  11. 2026Scheduling Thoughts: Learning the Order of Thought in Diffusion Language Modelscited by 1
  12. 2026From Interface to Inference: Eliciting Any-Order Inference from Any-Order Modelscited by 1
10 more
  1. 2026Learning Generation Orders for Masked Discrete Diffusion Models via Variational Inferencecited by 0
  2. 2026Progressive Refinement Regulation for Accelerating Diffusion Language Model Decodingcited by 0
  3. 2026Understanding and Accelerating the Training of Masked Diffusion Language Modelscited by 0
  4. 2026Adaptive Order Policies for Masked Diffusioncited by 0
  5. 2026One-step lowest-variance selection in a Gaussian random-field model motivated by masked diffusion: Total correlation and a square root collision thresholdcited by 0
  6. 2026DDSynth-RL: Audio Synthesizer Inversion via Discrete Diffusion with Reinforcement Learningcited by 0
  7. 2026Entropy Can Flow, or It Can Guide. Be Entropy. LEDFlow: Introducing Entropy-guided Generation Order into Uniform Discrete Flowcited by 0
  8. 2026Optimizing Denoising Trajectories in dLLMs: A Lightweight Evolutionary Heuristic Approachcited by 0
  9. 2026Does Uniform Discrete Diffusion Need Time?cited by 0
  10. 2026Batch-Paced: Fixed-Budget Batch Decoding in Discrete Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Plan for Speed: Dilated Scheduling for Masked Diffusion Language Models
  2. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking
  3. 2025Optimal Inference Schedules for Masked Diffusion Models
  4. 2025Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Masking
  5. 2025MDPO: Overcoming the Training-Inference Divide of Masked Diffusion Language Models
  6. 2026Not All Denoising Steps Are Equal: Model Scheduling for Faster Masked Diffusion Language Models
  7. 2025Lookahead Unmasking Elicits Accurate Decoding in Diffusion Language Models
  8. 2026Machine Unlearning for Masked Diffusion Language Models