mlsanityData to 7 Oct 2026

Informed Correctors for Discrete Diffusion Models

arXiv 2407.21243 ↗First posted 30 Jul 2024cs.LG cs.AIcited by 57

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Discrete diffusion has emerged as a powerful framework for generative modeling in discrete domains, yet efficiently sampling from these models remains challenging. Existing sampling strategies often struggle to balance computation and sample quality when the number of sampling steps is reduced, even when the model has learned the data distribution well. To address these limitations, we propose a predictor-corrector sampling scheme where the corrector is informed by the diffusion model to more reliably counter the accumulating approximation errors. To further enhance the effectiveness of our informed corrector, we introduce complementary architectural modifications based on hollow transformers and a simple tailored training objective that leverages more training signal. We use a synthetic example to illustrate the failure modes of existing samplers and show how informed correctors alleviate these problems. On the text8 and tokenized ImageNet 256x256 datasets, our informed corrector consistently produces superior samples with fewer errors or improved FID scores for discrete diffusion models. These results underscore the potential of informed correctors for fast and high-fidelity generation using discrete diffusion. Our code is available at https://github.com/lindermanlab/informed-correctors.

Key facts. Cited by 57 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 46 of its 69 references that are in the map. First posted on arXiv 30 Jul 2024.

Builds on

46 of 69 references.

Closest ancestors · 13

Same direction, newest first

  1. 2023Blackout Diffusion: Generative Diffusion Models in Discrete-State Spaces18 May
  2. 2022Score-based Continuous-time Discrete Diffusion Models30 Nov
  3. 2022From Denoising Diffusions to Denoising Markov Models7 Nov
  4. 2022A Continuous Time Framework for Discrete Denoising Models30 May
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  6. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesis29 Nov
  7. 2021Structured Denoising Diffusion Models in Discrete State-Spaces7 Jul
  8. 2021Cascaded Diffusion Models for High Fidelity Image Generation30 May
  9. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  10. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  11. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  12. 2020Denoising Diffusion Implicit Models6 Oct
1 more
  1. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 33

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  3. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  4. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  5. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  6. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  7. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  8. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  9. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 925
  10. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  11. 2022Autoregressive Image Generation using Residual Quantizationcited by 706
  12. 2021Vector-quantized Image Modeling with Improved VQGANcited by 656
21 more
  1. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  2. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  3. 2022DiGress: Discrete Denoising diffusion for graph generationcited by 503
  4. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 371
  5. 2024Discrete Flow Matchingcited by 301
  6. 2021Autoregressive Diffusion Modelscited by 208
  7. 2019Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacementcited by 205
  8. 2023A Reparameterized Discrete Diffusion Model for Text Generationcited by 152
  9. 2014A* Samplingcited by 150
  10. 2022Concrete Score Matching: Generalized Score Matching for Discrete Datacited by 144
  11. 2021Step-unrolled Denoising Autoencoders for Text Generationcited by 102
  12. 2021Oops I Took A Gradient: Scalable Sampling for Discrete Distributionscited by 92
  13. 2022Improved Masked Image Generation with Token-Criticcited by 63
  14. 2023Functional Flow Matchingcited by 63
  15. 2019Minimum Stein Discrepancy Estimatorscited by 57
  16. 2022A Langevin-like Sampler for Discrete Distributionscited by 42
  17. 2022Discrete Langevin Sampler via Wasserstein Gradient Flowcited by 23
  18. 2022Gradient Estimation with Discrete Stein Operatorscited by 22
  19. 2015The Hamming Ball Samplercited by 16
  20. 2023StraIT: Non-autoregressive Generation with Stratified Image Transformercited by 10
  21. 2017Improving Gibbs Sampler Scan Quality with DoGScited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 31; 2026: 24.

57 papers.

Sort:

By area:

  1. 2025Large Language Diffusion Modelscited by 872
  2. 2025Remasking Discrete Diffusion Models with Inference-Time Scalingcited by 181
  3. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmaskingcited by 128
  4. 2025The Diffusion Dualitycited by 124
  5. 2025Inference-Time Alignment in Diffusion Models with Reward-Guided Generation: Tutorial and Reviewcited by 82
  6. 2025Path Planning for Masked Diffusion Model Samplingcited by 79
  7. 2025Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithmscited by 47
  8. 2025KLASS: KL-Guided Fast Inference in Masked Diffusion Modelscited by 44
  9. 2024textitJump Your Steps: Optimizing Sampling Schedule of Discrete Diffusion Modelscited by 40
  10. 2025Fine-Tuning Masked Diffusion for Provable Self-Correctioncited by 39
  11. 2025Fine-Tuning Discrete Diffusion Models with Policy Gradient Methodscited by 36
  12. 2025Continuously Augmented Discrete Diffusion model for Categorical Generative Modelingcited by 32
38 more
  1. 2025Target Concrete Score Matching: A Holistic Framework for Discrete Diffusioncited by 25
  2. 2025Absorb and Converge: Provable Convergence Guarantee for Absorbing Discrete Diffusion Modelscited by 20
  3. 2025Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Maskingcited by 19
  4. 2025Ultra-Fast Language Generation via Discrete Diffusion Divergence Instructcited by 15
  5. 2025Reviving Any-Subset Autoregressive Models with Principled Parallel Sampling and Speculative Decodingcited by 14
  6. 2025Why Masking Diffusion Works: Condition on the Jump Schedule for Improved Discrete Diffusioncited by 14
  7. 2025Planner Aware Path Learning in Diffusion Language Models Trainingcited by 14
  8. 2025Error Bounds and Optimal Schedules for Masked Diffusions with Factorized Approximationscited by 13
  9. 2026Learn from Your Mistakes: Self-Correcting Masked Diffusion Modelscited by 12
  10. 2026Uniform Diffusion Models Revisited: Leave-One-Out Denoiser and Absorbing State Reformulationcited by 11
  11. 2025Simple and Critical Iterative Denoising: A Recasting of Discrete Diffusion in Graph Generationcited by 9
  12. 2025Learnable Sampler Distillation for Discrete Diffusion Modelscited by 8
  13. 2025Inference-Time Scaling of Diffusion Language Models via Trajectory Refinementcited by 7
  14. 2025A Diffusion Model to Shrink Proteins While Maintaining Their Functioncited by 6
  15. 2025Dimathtt[M]O: Distilling Masked Diffusion Models into One-step Generatorcited by 5
  16. 2025Plug-and-Play Context Feature Reuse for Efficient Masked Generationcited by 5
  17. 2025Guided Star-Shaped Masked Diffusioncited by 5
  18. 2024G2D2: Gradient-Guided Discrete Diffusion for Inverse Problem Solvingcited by 4
  19. 2026Early Decisions Matter: Proximity Bias and Initial Trajectory Shaping in Non-Autoregressive Diffusion Language Modelscited by 4
  20. 2025Masked Diffusion Models as Energy Minimizationcited by 3
  21. 2025Fine-Tuning Diffusion Models via Intermediate Distribution Shapingcited by 3
  22. 2025Masked Diffusion Models are Secretly Learned-Order Autoregressive Modelscited by 3
  23. 2025Diffusion Language Model Inference with Monte Carlo Tree Searchcited by 3
  24. 2026Improved Sampling Schedules for Discrete Diffusion Modelscited by 3
  25. 2026Reward-Guided Discrete Diffusion via Clean-Sample Markov Chain for Molecule and Biological Sequence Designcited by 3
  26. 2026Neural Continuous-Time Markov Chain: Discrete Diffusion via Decoupled Jump Timing and Directioncited by 3
  27. 2026Discrete-WAM: Unified Discrete Vision-Action Token Editing for World-Policy Learningcited by 3
  28. 2026Provably adaptive sampling with uniform and remasking discrete diffusion modelscited by 3
  29. 2025Simple Denoising Diffusion Language Modelscited by 2
  30. 2026Training-Free Self-Correction for Multimodal Masked Diffusion Modelscited by 2
  31. 2026A2D2: Fine-Tuning Any-Length Discrete Diffusion for Adaptive Decodingcited by 2
  32. 2026BackPlay: Head-Only Look-Back Self-Correction for Diffusion Language Modelscited by 1
  33. 2026Is Your Diffusion Sampler Actually Correct? A Sampler-Centric Evaluation of Discrete Diffusion Language Modelscited by 1
  34. 2026Generalized Discrete Diffusion with Self-Correctioncited by 1
  35. 2026From Scores to Gibbs Correctors: Accelerating Uniform-Rate Discrete Diffusion Modelscited by 1
  36. 2026Concurrent Image Understanding and Generation: Self-Correcting Coupled Markov Jump Processescited by 1
  37. 2026Stratified Hazard Sampling: Minimal-Variance Event Scheduling for CTMC/DTMC Discrete Diffusion and Flow Modelscited by 0
  38. 2026On the Error-Correcting Effects of Stochasticity in Discrete Diffusioncited by 0

50 most cited of 57 All 57 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026The Diffusion Duality, Chapter II: Psi-Samplers
  2. 2024DC-Solver: Improving Predictor-Corrector Diffusion Sampler via Dynamic Compensation
  3. 2022Progressive Distillation for Fast Sampling of Diffusion Models
  4. 2022On Distillation of Guided Diffusion Models
  5. 2024SFDDM: Single-fold Distillation for Diffusion models
  6. 2026Forward-Learned Discrete Diffusion: Learning how to noise to denoise faster
  7. 2023UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models
  8. 2026Interpolating Discrete Diffusion Models with Controllable Resampling