mlsanityData to 7 Oct 2026

Remasking Discrete Diffusion Models with Inference-Time Scaling

arXiv 2503.00307 ↗First posted 1 Mar 2025cs.LG stat.MLcited by 181

On the map: LLM reasoning → Diffusion reasoning and test-time scaling

also in 1 more
Abstract (arXiv)

Part of the success of diffusion models stems from their ability to perform iterative refinement, i.e., repeatedly correcting outputs during generation. However, modern masked discrete diffusion lacks this capability: when a token is generated, it cannot be updated again, even when it introduces an error. Here, we address this limitation by introducing the remasking diffusion model (ReMDM) sampler, a method that can be applied to pretrained masked diffusion models in a principled way and that is derived from a discrete diffusion model with a custom remasking backward process. Most interestingly, ReMDM endows discrete diffusion with a form of inference-time compute scaling. By increasing the number of sampling steps, ReMDM generates natural language outputs that approach the quality of autoregressive models, whereas when the computation budget is limited, ReMDM better maintains quality. ReMDM also improves sample quality of masked diffusion models for discretized images, and in scientific domains such as molecule design, ReMDM facilitates diffusion guidance and pushes the Pareto frontier of controllability relative to classical masking and uniform noise diffusion. We provide the code along with a blog post on the project page: https://guanghanwang.com/remdm

Key facts. Cited by 181 papers in the map (data through 7 Oct 2026). Placed in Diffusion reasoning and test-time scaling, one of 17 directions in LLM reasoning. Builds on 35 of its 45 references that are in the map. First posted on arXiv 1 Mar 2025.

Builds on

35 of 45 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Steps16 Jan

Other references · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  8. 2022Scalable Diffusion Models with Transformerscited by 5,931
  9. 2022Classifier-Free Diffusion Guidancecited by 5,717
  10. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  11. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  12. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
22 more
  1. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  2. 2016Improved Techniques for Training GANscited by 2,938
  3. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  4. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  5. 2023Consistency Modelscited by 1,778
  6. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  7. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  8. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  9. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
  10. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  11. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 466
  12. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  13. 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 374
  14. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 371
  15. 2024Discrete Flow Matchingcited by 301
  16. 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Designcited by 294
  17. 2018Molecular Transformer - A Model for Uncertainty-Calibrated Chemical Reaction Predictioncited by 225
  18. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Samplingcited by 204
  19. 2024Simple Guidance Mechanisms for Discrete Diffusion Modelscited by 148
  20. 2024Unlocking Guidance for Discrete State-Space Diffusion and Flow Modelscited by 124
  21. 2024Informed Correctors for Discrete Diffusion Modelscited by 57
  22. 2021Divergence Frontiers for Generative Models: Sample Complexity, Quantization Effects, and Frontier Integralscited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 68; 2026: 113.

181 papers.

Sort:

By area:

  1. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmaskingcited by 128
  2. 2025The Diffusion Dualitycited by 124
  3. 2025Accelerating Diffusion LLMs via Adaptive Parallel Decodingcited by 108
  4. 2025A Survey on Diffusion Language Modelscited by 75
  5. 2025wd1: Weighted Policy Optimization for Reasoning in Diffusion Language Modelscited by 67
  6. 2025Discrete Diffusion in Large Language and Multimodal Models: A Surveycited by 61
  7. 2026LLaDA2.1: Speeding Up Text Diffusion via Token Editingcited by 48
  8. 2025KLASS: KL-Guided Fast Inference in Masked Diffusion Modelscited by 44
  9. 2025ParallelBench: Understanding the Trade-offs of Parallel Decoding in Diffusion LLMscited by 42
  10. 2025Fine-Tuning Masked Diffusion for Provable Self-Correctioncited by 39
  11. 2025Don't Settle Too Early: Self-Reflective Remasking for Diffusion Language Modelscited by 37
  12. 2025ParaThinker: Native Parallel Thinking as a New Paradigm to Scale LLM Test-time Computecited by 36
38 more
  1. 2025Esoteric Language Models: A Family of Any-Order Diffusion LLMscited by 33
  2. 2025Time Is a Feature: Exploiting Temporal Dynamics in Diffusion Language Modelscited by 33
  3. 2025MDPO: Overcoming the Training-Inference Divide of Masked Diffusion Language Modelscited by 32
  4. 2025Continuously Augmented Discrete Diffusion model for Categorical Generative Modelingcited by 32
  5. 2025Multiverse: Your Language Models Secretly Decide How to Parallelize and Merge Generationcited by 30
  6. 2025AdaBlock-dLLM: Semantic-Aware Diffusion LLM Inference via Adaptive Block Sizecited by 30
  7. 2025Improving Reasoning for Diffusion Language Models via Group Diffusion Policy Optimizationcited by 30
  8. 2025Learning Unmasking Policies for Diffusion Language Modelscited by 29
  9. 2025A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Modelscited by 27
  10. 2025d2: Improving Reasoning in Diffusion Language Models via Trajectory Likelihood Estimationcited by 27
  11. 2026ELF: Embedded Language Flowscited by 27
  12. 2025Cosmos: Compressed and Smooth Latent Space for Text Diffusion Modelingcited by 24
  13. 2025Diffusion LLM with Native Variable Generation Lengths: Let [EOS] Lead the Waycited by 22
  14. 2025From Next-Token to Next-Block: A Principled Adaptation Path for Diffusion LLMscited by 22
  15. 2026Scaling Beyond Masked Diffusion Language Modelscited by 22
  16. 2025Encoder-Decoder Diffusion Language Models for Efficient Training and Inferencecited by 21
  17. 2025MDNS: Masked Diffusion Neural Sampler via Stochastic Optimal Controlcited by 20
  18. 2025Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Maskingcited by 19
  19. 2025Unifying Autoregressive and Diffusion-Based Sequence Generationcited by 17
  20. 2025Anchored Diffusion Language Modelcited by 17
  21. 2025DiFFPO: Training Diffusion LLMs to Reason Fast and Furious via Reinforcement Learningcited by 17
  22. 2026Confidence-Based Decoding is Provably Efficient for Diffusion Language Modelscited by 17
  23. 2025FS-DFM: Fast and Accurate Long Text Generation with Few-Step Diffusion Language Modelscited by 16
  24. 2025Soft-Masked Diffusion Language Modelscited by 15
  25. 2025Lookahead Unmasking Elicits Accurate Decoding in Diffusion Language Modelscited by 15
  26. 2025Planner Aware Path Learning in Diffusion Language Models Trainingcited by 14
  27. 2025Saber: An Efficient Sampling with Adaptive Acceleration and Backtracking Enhanced Remasking for Diffusion Language Modelcited by 13
  28. 2025Parallel Sampling from Masked Diffusion Models via Conditional Independence Testingcited by 13
  29. 2025Error Bounds and Optimal Schedules for Masked Diffusions with Factorized Approximationscited by 13
  30. 2025ScalingNoise: Scaling Inference-Time Search for Generating Infinite Videoscited by 12
  31. 2025Inference-Time Scaling of Discrete Diffusion Models via Importance Weighting and Optimal Proposal Designcited by 12
  32. 2025LLaDA-MedV: Exploring Large Language Diffusion Models for Biomedical Image Understandingcited by 12
  33. 2026Learn from Your Mistakes: Self-Correcting Masked Diffusion Modelscited by 12
  34. 2026The Diffusion Duality, Chapter II: Psi-Samplerscited by 12
  35. 2026dLLM: Simple Diffusion Language Modelingcited by 12
  36. 2025Corrective Diffusion Language Modelscited by 11
  37. 2026Uniform Diffusion Models Revisited: Leave-One-Out Denoiser and Absorbing State Reformulationcited by 11
  38. 2026Improving Diffusion Language Model Decoding through Joint Search in Generation Order and Token Spacecited by 10

50 most cited of 181 All 181 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Recursive Scaling in Masked Diffusion Models
  2. 2025Why Masking Diffusion Works: Condition on the Jump Schedule for Improved Discrete Diffusion
  3. 2026VGB for Masked Diffusion Model: Efficient Test-time Scaling for Reward Satisfaction and Sample Editing
  4. 2025Bringing Stability to Diffusion: Decomposing and Reducing Variance of Training Masked Diffusion Models
  5. 2025Path Planning for Masked Diffusion Model Sampling
  6. 2026CoDR: Training-Free Confidence-Drift Remasking for Diffusion Language Models
  7. 2026Simple Self-Conditioning Adaptation for Masked Diffusion Models
  8. 2024SFDDM: Single-fold Distillation for Diffusion models