mlsanityData to 7 Oct 2026

Fine-Tuning Masked Diffusion for Provable Self-Correction

arXiv 2510.01384 ↗First posted 1 Oct 2025cs.LGcited by 39

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

A natural desideratum for generative models is self-correction--detecting and revising low-quality tokens at inference. While Masked Diffusion Models (MDMs) have emerged as a promising approach for generative modeling in discrete spaces, their capacity for self-correction remains poorly understood. Prior attempts to incorporate self-correction into MDMs either require overhauling MDM architectures/training or rely on imprecise proxies for token quality, limiting their applicability. Motivated by this, we introduce PRISM--Plug-in Remasking for Inference-time Self-correction of Masked Diffusions--a lightweight, model-agnostic approach that applies to any pretrained MDM. Theoretically, PRISM defines a self-correction loss that provably learns per-token quality scores, without RL or a verifier. These quality scores are computed in the same forward pass with MDM and used to detect low-quality tokens. Empirically, PRISM advances MDM inference across domains and scales: Sudoku; unconditional text (170M); and code with LLaDA (8B).

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 40 of its 51 references that are in the map. First posted on arXiv 1 Oct 2025.

Builds on

40 of 51 references.

Closest ancestors · 19

Same direction, newest first

  1. 2025Don't Settle Too Early: Self-Reflective Remasking for Diffusion Language Models28 Sep
  2. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation25 Jun
  3. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking30 May
  4. 2025Anchored Diffusion Language Model24 May
  5. 2025Generalized Interpolating Discrete Diffusion6 Mar
  6. 2025Remasking Discrete Diffusion Models with Inference-Time Scaling1 Mar
  7. 2025Large Language Diffusion Models14 Feb
  8. 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions10 Feb
  9. 2025Path Planning for Masked Diffusion Model Sampling5 Feb
  10. 2024Energy-Based Diffusion Language Models for Text Generation28 Oct
  11. 2024Think While You Generate: Discrete Diffusion with Planned Denoising8 Oct
  12. 2024Discrete Copula Diffusion2 Oct
7 more
  1. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling4 Sep
  2. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  3. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  4. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun
  5. 2024Glauber Generative Model: Discrete Diffusion Models via Binary Classification27 May
  6. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct
  7. 2023A Reparameterized Discrete Diffusion Model for Text Generation11 Feb

Other references · 21

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2022Scalable Diffusion Models with Transformerscited by 5,931
  5. 2021Program Synthesis with Large Language Modelscited by 3,509
  6. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  7. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  8. 2022Building Normalizing Flows with Stochastic Interpolantscited by 964
  9. 2023Stochastic Interpolants: A Unifying Framework for Flows and Diffusionscited by 897
  10. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
  11. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  12. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 466
9 more
  1. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  2. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 405
  3. 2024Discrete Flow Matchingcited by 301
  4. 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Designcited by 294
  5. 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 225
  6. 2024OpenCoder: The Open Cookbook for Top-Tier Code Large Language Modelscited by 127
  7. 2022Improved Masked Image Generation with Token-Criticcited by 63
  8. 2024Informed Correctors for Discrete Diffusion Modelscited by 57
  9. 2025RIV: Recursive Introspection Mask Diffusion Vision Language Modelcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 36.

39 papers.

Sort:

By area:

  1. 2025ParallelBench: Understanding the Trade-offs of Parallel Decoding in Diffusion LLMscited by 42
  2. 2026Language Modeling with Hyperspherical Flowscited by 15
  3. 2026Learn from Your Mistakes: Self-Correcting Masked Diffusion Modelscited by 12
  4. 2026The Diffusion Duality, Chapter II: Psi-Samplerscited by 12
  5. 2025Corrective Diffusion Language Modelscited by 11
  6. 2026CORE: Context-Robust Remasking for Diffusion Language Modelscited by 10
  7. 2025Guided Star-Shaped Masked Diffusioncited by 5
  8. 2026Flow Reasoning Models: Turning Flows Into Efficient Recurrent Reasonerscited by 5
  9. 2026Expert-Choice Routing Enables Adaptive Computation in Diffusion Language Modelscited by 3
  10. 2026Provably adaptive sampling with uniform and remasking discrete diffusion modelscited by 3
  11. 2026Training-Free Self-Correction for Multimodal Masked Diffusion Modelscited by 2
  12. 2026Locally Coherent Parallel Decoding in Diffusion Language Modelscited by 2
27 more
  1. 2026A2D2: Fine-Tuning Any-Length Discrete Diffusion for Adaptive Decodingcited by 2
  2. 2026Don't Commit Alone: Joint Token Commitment in Diffusion Language Modelscited by 2
  3. 2026BackPlay: Head-Only Look-Back Self-Correction for Diffusion Language Modelscited by 1
  4. 2026Generalized Discrete Diffusion with Self-Correctioncited by 1
  5. 2026Latent-DARM: Bridging Discrete Diffusion And Autoregressive Models For Reasoningcited by 1
  6. 2026Self-Aware Markov Models for Discrete Reasoningcited by 1
  7. 2026Remask, Don't Replace: Token-to-Mask Refinement in Diffusion Language Modelscited by 1
  8. 2026The Confidence Shortcut: A Reasoning Failure Mode of Masked Diffusion Modelscited by 1
  9. 2026DSL-LLaDA: Scaling Continuous Denoising to 8B Masked Diffusion LMscited by 1
  10. 2026Revise, Don't Freeze: Sampler-Matched Training for Self-Correcting Masked Diffusion Language Modelscited by 1
  11. 2026Re-evaluating Confidence Remasking in Masked Diffusion Language Modelscited by 1
  12. 2026VGB for Masked Diffusion Model: Efficient Test-time Scaling for Reward Satisfaction and Sample Editingcited by 1
  13. 2026Concurrent Image Understanding and Generation: Self-Correcting Coupled Markov Jump Processescited by 1
  14. 2026CaRE Compute-aware Remasking Evaluation Protocol for Masked Diffusion Language Modelscited by 1
  15. 2026Reconsidering Positional Supervision in Masked Diffusion Language Model Trainingcited by 0
  16. 2026Targeted Remasking: Replacing Token Editing with Token-to-Mask Refinement in Discrete Diffusion Language Modelscited by 0
  17. 2026On the Error-Correcting Effects of Stochasticity in Discrete Diffusioncited by 0
  18. 2026BlockGen: Flexible Blockwise Sequence Modeling with Hybrid Samplerscited by 0
  19. 2026NAVIRA: Decoupled Stochastic Remasking for Masked Diffusion Language Modelscited by 0
  20. 2026Forward-Free Diffusion Language Models with BPTT-Free Looped Refinementcited by 0
  21. 2026Detect, Remask, Repair: Diffusion Editing for Faithful Summarization of Evolving Contextscited by 0
  22. 2026JUMP: Single-Pass Membership Inference on Fine-Tuned Diffusion Language Modelscited by 0
  23. 2026From Interface to Inference: Eliciting Any-Order Inference from Any-Order Modelscited by 0
  24. 2026A Ticket from Marginals to Joints: Coupled-Noise Distillation for One-Step Block Generation in Diffusion Language Modelscited by 0
  25. 2026LOCKR: A Hidden-State Trajectory-Guided Planner for Detecting and Repairing Stable-but-Wrong Lock-In in Diffusion Language Modelscited by 0
  26. 2026Grab a Coffee: Future-Aware Guidance for Discrete Diffusion with Compiled Objectivescited by 0
  27. 2026Blackboard Intelligence Can Surpass Autoregressive on Globally Constrained Problemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Masking
  2. 2025Variational Masked Diffusion Models
  3. 2025Improving Discrete Diffusion Unmasking Policies Beyond Explicit Reference Policies
  4. 2026Stop Training for the Worst: Progressive Unmasking Accelerates Masked Diffusion Training
  5. 2026Let It Go or Learn to Self-Correct: Continuous Diffusion for Constrained Discrete Tasks
  6. 2026Pivot-SD: Efficient Self-Distillation for Masked Diffusion Language Models
  7. 2026Multi-Turn Reflective Masking Elicits Reasoning in Mask Diffusion Models
  8. 2026Know When to Hold 'em: Correct-Token Retention in Uniform-State Diffusion Language Models