mlsanityData to 7 Oct 2026

Learning Unmasking Policies for Diffusion Language Models

arXiv 2512.09106 ↗First posted 9 Dec 2025cs.LGcited by 29

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Diffusion (Large) Language Models (dLLMs) now match the downstream performance of their autoregressive counterparts on many tasks, while holding the promise of being more efficient during inference. One critical design aspect of dLLMs is the sampling procedure that selects which tokens to unmask at each diffusion step. Indeed, recent work has found that heuristic strategies such as confidence thresholding improve both sample quality and token throughput compared to random unmasking. However, such heuristics have downsides: they require manual tuning, and we observe that their performance degrades with larger block sizes. In this work, we instead propose to train sampling procedures using reinforcement learning. Specifically, we formalize masked diffusion sampling as a Markov decision process in which the dLLM serves as the environment, and propose a lightweight policy based on a single-layer transformer that maps dLLM token confidences to unmasking decisions. Our experiments show that these trained policies match the performance of state-of-the-art heuristics when combined with semi-autoregressive (block) generation, while outperforming them in the full-diffusion setting.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 72 of its 87 references that are in the map. First posted on arXiv 9 Dec 2025.

Builds on

72 of 87 references.

Closest ancestors · 34

Same direction, newest first

  1. 2025KLASS: KL-Guided Fast Inference in Masked Diffusion Models7 Nov
  2. 2025Encoder-Decoder Diffusion Language Models for Efficient Training and Inference26 Oct
  3. 2025Parallel Sampling from Masked Diffusion Models via Conditional Independence Testing24 Oct
  4. 2025Saber: An Efficient Sampling with Adaptive Acceleration and Backtracking Enhanced Remasking for Diffusion Language Model20 Oct
  5. 2025SPG: Sandwiched Policy Gradient for Masked Diffusion Language Models10 Oct
  6. 2025dInfer: An Efficient Inference Framework for Diffusion Language Models9 Oct
  7. 2025Improving Reasoning for Diffusion Language Models via Group Diffusion Policy Optimization9 Oct
  8. 2025Improving Discrete Diffusion Unmasking Policies Beyond Explicit Reference Policies7 Oct
  9. 2025Simple Policy Gradients for Reasoning with Diffusion Language Models5 Oct
  10. 2025Self-Speculative Masked Diffusions4 Oct
  11. 2025DiFFPO: Training Diffusion LLMs to Reason Fast and Furious via Reinforcement Learning2 Oct
  12. 2025Don't Settle Too Early: Self-Reflective Remasking for Diffusion Language Models28 Sep
22 more
  1. 2025Revolutionizing Reinforcement Learning Framework for Diffusion Large Language Models8 Sep
  2. 2025Diffusion Language Models Know the Answer Before Decoding27 Aug
  3. 2025Dream 7B: Diffusion Large Language Models21 Aug
  4. 2025Empirical Analysis of Decoding Biases in Masked Diffusion Models18 Aug
  5. 2025Time Is a Feature: Exploiting Temporal Dynamics in Diffusion Language Models12 Aug
  6. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing8 Aug
  7. 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference4 Aug
  8. 2025wd1: Weighted Policy Optimization for Reasoning in Diffusion Language Models7 Jul
  9. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation25 Jun
  10. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking30 May
  11. 2025Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decoding22 May
  12. 2025Reinforcing the Diffusion Chain of Lateral Thought with Diffusion Language Models15 May
  13. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
  14. 2025Remasking Discrete Diffusion Models with Inference-Time Scaling1 Mar
  15. 2025Large Language Diffusion Models14 Feb
  16. 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions10 Feb
  17. 2024Think While You Generate: Discrete Diffusion with Planned Denoising8 Oct
  18. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  19. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  20. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun
  21. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct
  22. 2022Continuous diffusion for categorical data28 Nov

Other references · 38

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  8. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  9. 2021Program Synthesis with Large Language Modelscited by 3,509
  10. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  11. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  12. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 591
26 more
  1. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  2. 2016Adaptive Computation Time for Recurrent Neural Networkscited by 479
  3. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
  4. 2017SkipNet: Learning Dynamic Routing in Convolutional Networkscited by 328
  5. 2022Confident Adaptive Language Modelingcited by 297
  6. 2023Understanding Diffusion Objectives as the ELBO with Simple Data Augmentationcited by 293
  7. 2019Monte Carlo Gradient Estimation in Machine Learningcited by 281
  8. 2025Training Language Models to Reason Efficientlycited by 242
  9. 2025d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learningcited by 178
  10. 2015Conditional Computation in Neural Networks for faster modelscited by 166
  11. 2025KodCode: A Diverse, Challenging, and Verifiable Synthetic Dataset for Codingcited by 115
  12. 2025Accelerating Diffusion LLMs via Adaptive Parallel Decodingcited by 108
  13. 2025S-GRPO: Early Exit via Reinforcement Learning in Reasoning Modelscited by 87
  14. 2024AceMath: Advancing Frontier Math Reasoning with Post-Training and Reward Modelingcited by 76
  15. 2025ShorterBetter: Guiding Reasoning Models to Find Optimal Inference Length for Efficient Reasoningcited by 70
  16. 2025dParallel: Learnable Parallel Decoding for dLLMscited by 59
  17. 2025Unified Multimodal Discrete Diffusioncited by 46
  18. 2025Wide-In, Narrow-Out: Revokable Decoding for Efficient and Effective DLLMscited by 45
  19. 2025Beyond Fixed: Training-Free Variable-Length Denoising for Diffusion Large Language Modelscited by 32
  20. 2025d2Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Cachingcited by 24
  21. 2025Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Modelscited by 19
  22. 2025Enhancing Reasoning for Diffusion LLMs via Distribution Matching Policy Optimizationcited by 15
  23. 2025Reviving Any-Subset Autoregressive Models with Principled Parallel Sampling and Speculative Decodingcited by 14
  24. 2025Enabling Approximate Joint Sampling in Diffusion LMscited by 9
  25. 2025Boundary-Guided Policy Optimization for Memory-efficient RL of Diffusion Large Language Modelscited by 9
  26. 2025Improving the Throughput of Diffusion-based Large Language Models via a Training-Free Confidence-Aware Calibrationcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 27.

29 papers.

Sort:

By area:

  1. 2025dUltra: Ultra-Fast Diffusion Language Models via Reinforcement Learningcited by 10
  2. 2026Adaptation to Intrinsic Dependence in Diffusion Language Modelscited by 9
  3. 2026Stop Training for the Worst: Progressive Unmasking Accelerates Masked Diffusion Trainingcited by 8
  4. 2026DUEL: Exact Likelihood for Masked Diffusion via Deterministic Unmaskingcited by 8
  5. 2026Dependency-Guided Parallel Decoding in Discrete Diffusion Language Modelscited by 8
  6. 2025Activation Steering for Masked Diffusion Language Modelscited by 7
  7. 2026The Design Space of Tri-Modal Masked Diffusion Modelscited by 6
  8. 2026A Tale of Two Temperatures: Simple, Efficient, and Diverse Sampling from Diffusion Language Modelscited by 5
  9. 2026ILRR: Inference-Time Steering Method for Masked Diffusion Language Modelscited by 3
  10. 2026TABES: Trajectory-Aware Backward-on-Entropy Steering for Masked Diffusion Modelscited by 3
  11. 2026DyLLM: Efficient Diffusion LLM Inference via Saliency-based Token Selection and Partial Attentioncited by 3
  12. 2026Fixed-Point Masked Generative Modelingcited by 2
17 more
  1. 2026Attention-Discounted Adaptive Sampler for Masked Diffusion Language Modelscited by 2
  2. 2026Mask-Aware Policy Gradients for Diffusion Language Modelscited by 2
  3. 2026Tuning the Implicit Regularizer of Masked Diffusion Language Models: Enhancing Generalization via Insights from k-Paritycited by 1
  4. 2026DPRM: A Plug-in Doob h transform-induced Token-Ordering Module for Discrete Diffusion Modelscited by 1
  5. 2026Visual-Redundancy-Controlled Parallel Decoding for Diffusion-Based Multimodal Large Language Modelscited by 1
  6. 2026Re-evaluating Confidence Remasking in Masked Diffusion Language Modelscited by 1
  7. 2026Scheduling Thoughts: Learning the Order of Thought in Diffusion Language Modelscited by 1
  8. 2026Accelerating Masked Diffusion Large Language Models: A Survey of Efficient Inference Techniquescited by 1
  9. 2026Temporal Self-Distillation: Faster Inference in Discrete Diffusion Language Modelscited by 1
  10. 2026Reconsidering Positional Supervision in Masked Diffusion Language Model Trainingcited by 0
  11. 2026Progressive Refinement Regulation for Accelerating Diffusion Language Model Decodingcited by 0
  12. 2026Adaptive Order Policies for Masked Diffusioncited by 0
  13. 2026One-step lowest-variance selection in a Gaussian random-field model motivated by masked diffusion: Total correlation and a square root collision thresholdcited by 0
  14. 2026From Interface to Inference: Eliciting Any-Order Inference from Any-Order Modelscited by 0
  15. 2026Prefix-Denoising Consistency: Test-Time Verification for Diffusion Language Modelscited by 0
  16. 2026Fork-dLLM: Avoiding the Flexibility Trap in Diffusion Language Modelscited by 0
  17. 2026ITC-MoE: Importance-guided Token-aware Compression for MoE Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025MDPO: Overcoming the Training-Inference Divide of Masked Diffusion Language Models
  2. 2026Reinforcement Learning for Diffusion LLMs with Entropy-Guided Step Selection and Stepwise Advantages
  3. 2025Optimal Inference Schedules for Masked Diffusion Models
  4. 2026LFPO: Likelihood-Free Policy Optimization for Masked Diffusion Models
  5. 2025Plan for Speed: Dilated Scheduling for Masked Diffusion Language Models
  6. 2026UnMaskFork: Test-Time Scaling for Masked Diffusion via Deterministic Action Branching
  7. 2026Where-to-Unmask: Ground-Truth-Guided Unmasking Order Learning for Masked Diffusion Language Models
  8. 2025d-TreeRPO: Towards More Reliable Policy Optimization for Diffusion Language Models