mlsanityData to 7 Oct 2026

A Continuous Time Framework for Discrete Denoising Models

arXiv 2205.14987 ↗First posted 30 May 2022stat.ML cs.LGcited by 437

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

We provide the first complete continuous time framework for denoising diffusion models of discrete data. This is achieved by formulating the forward noising process and corresponding reverse time generative process as Continuous Time Markov Chains (CTMCs). The model can be efficiently trained using a continuous time version of the ELBO. We simulate the high dimensional CTMC using techniques developed in chemical physics and exploit our continuous time framework to derive high performance samplers that we show can outperform discrete time methods for discrete data. The continuous time treatment also enables us to derive a novel theoretical result bounding the error between the generated sample distribution and the true data distribution.

Key facts. Cited by 437 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 32 of its 47 references that are in the map. First posted on arXiv 30 May 2022.

Builds on

32 of 47 references.

Closest ancestors · 11

Same direction, newest first

  1. 2022Fast Sampling of Diffusion Models with Exponential Integrator29 Apr
  2. 2022Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models17 Jan
  3. 2021Score-Based Generative Modeling with Critically-Damped Langevin Diffusion14 Dec
  4. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesis29 Nov
  5. 2021Structured Denoising Diffusion Models in Discrete State-Spaces7 Jul
  6. 2021Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modeling1 Jun
  7. 2021Gotta Go Fast When Generating Data with Score-Based Models28 May
  8. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  9. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  10. 2020Denoising Diffusion Implicit Models6 Oct
  11. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 21

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  4. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  5. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  6. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 935
  7. 2022Equivariant Diffusion for Molecule Generation in 3Dcited by 666
  8. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  9. 2017PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modificationscited by 415
  10. 2021Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models for Inverse Problems through Stochastic Contractioncited by 306
  11. 2021Autoregressive Diffusion Modelscited by 208
  12. 2019Effectively Unbiased FID and Inception Score and where to find themcited by 161
9 more
  1. 2021Symbolic Music Generation with Diffusion Modelscited by 152
  2. 2017MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompanimentcited by 151
  3. 2017PixelSNAIL: An Improved Autoregressive Generative Modelcited by 137
  4. 2021ImageBART: Bidirectional Context with Multinomial Diffusion for Autoregressive Image Synthesiscited by 122
  5. 2021Step-unrolled Denoising Autoencoders for Text Generationcited by 102
  6. 2017Variational Walkback: Learning a Transition Operator as a Stochastic Recurrent Netcited by 30
  7. 2021Beyond In-Place Corruption: Insertion and Deletion In Denoising Probabilistic Modelscited by 23
  8. 2022Diffusion bridges vector quantized Variational AutoEncoderscited by 16
  9. 2021Generative Probabilistic Image Colorizationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 10; 2023: 23; 2024: 58; 2025: 165; 2026: 181.

437 papers.

Sort:

By area:

  1. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  2. 2022Diffusion Models in Vision: A Surveycited by 1,008
  3. 2025Large Language Diffusion Modelscited by 872
  4. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  5. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
  6. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
  7. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  8. 2025Dream 7B: Diffusion Large Language Modelscited by 515
  9. 2022DiGress: Discrete Denoising diffusion for graph generationcited by 503
  10. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
  11. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 385
  12. 2022TabDDPM: Modelling Tabular Data with Diffusion Modelscited by 373
38 more
  1. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 371
  2. 2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioningcited by 362
  3. 2024Discrete Flow Matchingcited by 301
  4. 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Designcited by 294
  5. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 273
  6. 2022A Survey on Generative Diffusion Modelcited by 238
  7. 2023DIFUSCO: Graph-based Diffusion Solvers for Combinatorial Optimizationcited by 216
  8. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Samplingcited by 204
  9. 2022Continuous diffusion for categorical datacited by 202
  10. 2024Scaling up Masked Diffusion Models on Textcited by 195
  11. 2025Remasking Discrete Diffusion Models with Inference-Time Scalingcited by 181
  12. 2022Score-based Continuous-time Discrete Diffusion Modelscited by 167
  13. 2024Improving Diffusion Inverse Problem Solving with Decoupled Noise Annealingcited by 157
  14. 2023A Reparameterized Discrete Diffusion Model for Text Generationcited by 152
  15. 2024Simple Guidance Mechanisms for Discrete Diffusion Modelscited by 148
  16. 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 145
  17. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planningcited by 135
  18. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 128
  19. 2024Derivative-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value-Based Decodingcited by 127
  20. 2024Unlocking Guidance for Discrete State-Space Diffusion and Flow Modelscited by 124
  21. 2025The Diffusion Dualitycited by 124
  22. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcingcited by 111
  23. 2024Dirichlet Flow Matching with Applications to DNA Sequence Designcited by 110
  24. 2023Copilot4D: Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusioncited by 105
  25. 2025Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decodingcited by 101
  26. 2024Understanding Reinforcement Learning-Based Fine-Tuning of Diffusion Models: A Tutorial and Reviewcited by 92
  27. 2024Energy-Based Diffusion Language Models for Text Generationcited by 86
  28. 2025Dream-Coder 7B: An Open Diffusion Language Model for Codecited by 81
  29. 2025Path Planning for Masked Diffusion Model Samplingcited by 79
  30. 2023Dirichlet Diffusion Score Model for Biological Sequence Generationcited by 73
  31. 2024Discrete-state Continuous-time Diffusion for Graph Generationcited by 73
  32. 2024Beyond Autoregression: Fast LLMs via Self-Distillation Through Timecited by 73
  33. 2022HouseDiffusion: Vector Floorplan Generation via a Diffusion Model with Discrete and Continuous Denoisingcited by 72
  34. 2024An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimizationcited by 72
  35. 2024Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Designcited by 72
  36. 2024Amortizing intractable inference in diffusion models for vision, language, and controlcited by 70
  37. 2025Generalized Interpolating Discrete Diffusioncited by 68
  38. 2025Diffusion Language Models Know the Answer Before Decodingcited by 68

50 most cited of 437 All 437 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Interleaved Gibbs Diffusion: Generating Discrete-Continuous Data with Implicit Constraints
  2. 2025A Unified Approach to Analysis and Design of Denoising Markov Models
  3. 2023The Missing U for Efficient Diffusion Models
  4. 2024Structural and Convergence Analysis of Discrete-Time Denoising Diffusion Probabilistic Models
  5. 2026Efficient Sampling with Discrete Diffusion Models: Sharp and Adaptive Guarantees
  6. 2026A Tutorial on Diffusion Theory: From Differential Equations to Diffusion Models
  7. 2026Forward-Learned Discrete Diffusion: Learning how to noise to denoise faster
  8. 2022Diffusion Models for Graphs Benefit From Discrete State Spaces