mlsanityData to 7 Oct 2026

Corrective Diffusion Language Models

arXiv 2512.15596 ↗First posted 17 Dec 2025cs.LGcited by 11

On the map: Diffusion models → Discrete and text diffusion

also in 1 more
Abstract (arXiv)

While Diffusion Language Models (DLMs) are theoretically well-suited for iterative refinement due to their non-causal structure, they often fail to reliably revise incorrect tokens in practice. The key challenge lies in the model's inability to distinguish between correct and erroneous tokens in a visible sequence. Standard masked diffusion language model (MDLM) training is restricted to the objective of unmasking, undermining the effectiveness of refinement guided by confidence. Based on this observation, we study corrective behavior in DLMs, defined as the ability to assign lower confidence to incorrect tokens and iteratively refine them while preserving correct content. We show that this capability is not induced by conventional masked diffusion objectives and propose a post-training principle oriented by correction that explicitly supervises visible incorrect tokens, enabling discriminative confidence and targeted refinement. To evaluate corrective behavior, we introduce the Code Revision Benchmark, a controllable and executable benchmark for assessing error localization and in-place correction. Experiments on code revision tasks and parallel decoding scenarios demonstrate that models trained with our approach substantially outperform standard MDLMs, with gains that are most pronounced when parallel decoding introduces substantial uncertainty and iterative refinement becomes essential. Our code is publicly available at https://github.com/zhangshuibai/CDLM.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 36 of its 52 references that are in the map. First posted on arXiv 17 Dec 2025.

Builds on

36 of 52 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025Optimal Inference Schedules for Masked Diffusion Models6 Nov
  2. 2025Fine-Tuning Masked Diffusion for Provable Self-Correction1 Oct
  3. 2025Training Optimal Large Diffusion Language Models28 Sep
  4. 2025Dream 7B: Diffusion Large Language Models21 Aug
  5. 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference4 Aug
  6. 2025Flexible-length Text Infilling for Discrete Diffusion Models16 Jun
  7. 2025Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Masking24 May
  8. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
  9. 2025Generalized Interpolating Discrete Diffusion6 Mar
  10. 2025Remasking Discrete Diffusion Models with Inference-Time Scaling1 Mar
  11. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  12. 2022DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models28 Nov

Other references · 24

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  4. 2021Program Synthesis with Large Language Modelscited by 3,509
  5. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  6. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  7. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  8. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  9. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
  10. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
  11. 2024Simple Guidance Mechanisms for Discrete Diffusion Modelscited by 148
  12. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 128
12 more
  1. 2025Accelerating Diffusion LLMs via Adaptive Parallel Decodingcited by 108
  2. 2025ParallelBench: Understanding the Trade-offs of Parallel Decoding in Diffusion LLMscited by 42
  3. 2024BugsInPy: A Database of Existing Bugs in Python Programs to Enable Controlled Testing and Debugging Studiescited by 35
  4. 2025Self Speculative Decoding for Diffusion Large Language Modelscited by 29
  5. 2025Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Modelscited by 19
  6. 2025EDIT-Bench: Evaluating LLM Abilities to Perform Real-World Instructed Code Editscited by 13
  7. 2025RefineBench: Evaluating Refinement Capability of Language Models via Checklistscited by 13
  8. 2025RealCritic: Towards Effectiveness-Driven Evaluation of Language Model Critiquescited by 12
  9. 2025CreditDecoding: Accelerating Parallel Decoding in Diffusion Large Language Models with Trace Creditcited by 11
  10. 2025Effective Test-Time Scaling of Discrete Diffusion through Iterative Refinementcited by 7
  11. 2025LLM-Based Detection of Tangled Code Changes for Higher-Quality Method-Level Bug Datasetscited by 3
  12. 2024LecPrompt: A Prompt-based Approach for Logical Error Correction with CodeBERTcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 11.

11 papers.

  1. 2026The Flexibility Trap: Rethinking the Value of Arbitrary Order in Diffusion Language Modelscited by 16
  2. 2026DMax: Aggressive Parallel Decoding for dLLMscited by 14
  3. 2026Locally Coherent Parallel Decoding in Diffusion Language Modelscited by 2
  4. 2026Reasoning on the Manifold: Bidirectional Consistency for Self-Verification in Diffusion Language Modelscited by 2
  5. 2026Distilled Continuous Diffusion Language Models Can Write Code in Few Steps---or Onecited by 2
  6. 2026Edit-Based Refinement for Parallel Masked Diffusion Language Modelscited by 1
  7. 2026The Confidence Shortcut: A Reasoning Failure Mode of Masked Diffusion Modelscited by 1
  8. 2026Reconsidering Positional Supervision in Masked Diffusion Language Model Trainingcited by 0
  9. 2026Controllable Accent Normalization via Discrete Diffusioncited by 0
  10. 2026Forward-Free Diffusion Language Models with BPTT-Free Looped Refinementcited by 0
  11. 2026Nemotron-Labs-Diffusion-Image: Advancing Masked Discrete Diffusion for High-Resolution Image Synthesiscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Remask, Don't Replace: Token-to-Mask Refinement in Diffusion Language Models
  2. 2026CoDR: Training-Free Confidence-Drift Remasking for Diffusion Language Models
  3. 2026Re-evaluating Confidence Remasking in Masked Diffusion Language Models
  4. 2026Beyond Masks: Efficient, Flexible Diffusion Language Models via Deletion-Insertion Processes
  5. 2026CORE: Context-Robust Remasking for Diffusion Language Models
  6. 2026Revise, Don't Freeze: Sampler-Matched Training for Self-Correcting Masked Diffusion Language Models
  7. 2025Finish First, Perfect Later: Test-Time Token-Level Cross-Validation for Diffusion Large Language Models
  8. 2026Diffusion Language Models: An Experimental Analysis