mlsanityData to 14 Oct 2026

Score-based Continuous-time Discrete Diffusion Models

arXiv 2211.16750 ↗First posted 30 Nov 2022cs.LGcited by 169

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Score-based modeling through stochastic differential equations (SDEs) has provided a new perspective on diffusion models, and demonstrated superior performance on continuous data. However, the gradient of the log-likelihood function, i.e., the score function, is not properly defined for discrete spaces. This makes it non-trivial to adapt textcolorcdiffthe score-based modeling to categorical data. In this paper, we extend diffusion models to discrete variables by introducing a stochastic jump process where the reverse process denoises via a continuous-time Markov chain. This formulation admits an analytical simulation during backward sampling. To learn the reverse process, we extend score matching to general categorical data and show that an unbiased estimator can be obtained via simple matching of the conditional marginal distributions. We demonstrate the effectiveness of the proposed method on a set of synthetic and real-world music and image benchmarks.

Key facts. Cited by 169 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 25 of its 39 references that are in the map. First posted on arXiv 30 Nov 2022.

Builds on

25 of 39 references.

Closest ancestors · 9

Same direction, newest first

  1. 2022A Continuous Time Framework for Discrete Denoising Models30 May
  2. 2022Fast Sampling of Diffusion Models with Exponential Integrator29 Apr
  3. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesis29 Nov
  4. 2021Structured Denoising Diffusion Models in Discrete State-Spaces7 Jul
  5. 2021Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modeling1 Jun
  6. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  7. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  8. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  9. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 16

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  4. 2017Neural Discrete Representation Learningcited by 4,712
  5. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  6. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  7. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  8. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 574
  9. 2015MADE: Masked Autoencoder for Distribution Estimationcited by 307
  10. 2021Autoregressive Diffusion Modelscited by 210
  11. 2017MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompanimentcited by 151
  12. 2022Generative Flow Networks for Discrete Probabilistic Modelingcited by 114
4 more
  1. 2020Learning Discrete Energy-based Models via Auxiliary-variable Local Explorationcited by 30
  2. 2021Beyond In-Place Corruption: Insertion and Deletion In Denoising Probabilistic Modelscited by 23
  3. 2019Neural Networks with Cheap Differential Operatorscited by 18
  4. 2022Diffusion bridges vector quantized Variational AutoEncoderscited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 13; 2024: 40; 2025: 64; 2026: 52.

169 papers.

Sort:

By area:

  1. 2025Large Language Diffusion Modelscited by 912
  2. 2024Simple and Effective Masked Diffusion Language Modelscited by 848
  3. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 745
  4. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 626
  5. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 420
  6. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 409
  7. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 381
  8. 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Designcited by 305
  9. 2023DIFUSCO: Graph-based Diffusion Solvers for Combinatorial Optimizationcited by 221
  10. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Samplingcited by 208
  11. 2024Scaling up Masked Diffusion Models on Textcited by 199
  12. 2024Simple Guidance Mechanisms for Discrete Diffusion Modelscited by 155
38 more
  1. 2023A Reparameterized Discrete Diffusion Model for Text Generationcited by 152
  2. 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 148
  3. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planningcited by 135
  4. 2024Unlocking Guidance for Discrete State-Space Diffusion and Flow Modelscited by 129
  5. 2025Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decodingcited by 101
  6. 2025Inference-Time Alignment in Diffusion Models with Reward-Guided Generation: Tutorial and Reviewcited by 88
  7. 2025Path Planning for Masked Diffusion Model Samplingcited by 80
  8. 2024Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Designcited by 79
  9. 2024Discrete-state Continuous-time Diffusion for Graph Generationcited by 76
  10. 2023Dirichlet Diffusion Score Model for Biological Sequence Generationcited by 75
  11. 2024An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimizationcited by 73
  12. 2024Amortizing intractable inference in diffusion models for vision, language, and controlcited by 72
  13. 2025Discrete Diffusion in Large Language and Multimodal Models: A Surveycited by 61
  14. 2024Informed Correctors for Discrete Diffusion Modelscited by 57
  15. 2024Discrete Copula Diffusioncited by 54
  16. 2024How Discrete and Continuous Diffusion Meet: Comprehensive Analysis of Discrete Diffusion Models via a Stochastic Integral Frameworkcited by 48
  17. 2024Distillation of Discrete Diffusion through Dimensional Correlationscited by 48
  18. 2025Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithmscited by 47
  19. 2025FUDOKI: Discrete Flow-based Unified Understanding and Generation via Kinetic-Optimal Velocitiescited by 46
  20. 2024Convergence Analysis of Discrete Diffusion Model: Exact Implementation through Uniformizationcited by 44
  21. 2026Categorical Flow Mapscited by 44
  22. 2024Accelerating Diffusion Models with Parallel Sampling: Inference at Sub-Linear Time Complexitycited by 42
  23. 2025SPG: Sandwiched Policy Gradient for Masked Diffusion Language Modelscited by 41
  24. 2023Sampling, Diffusions, and Stochastic Localizationcited by 40
  25. 2024textitJump Your Steps: Optimizing Sampling Schedule of Discrete Diffusion Modelscited by 40
  26. 2023MissDiff: Training Diffusion Models on Tabular Data with Missing Valuescited by 39
  27. 2025Fine-Tuning Discrete Diffusion Models with Policy Gradient Methodscited by 38
  28. 2025CANDI: Hybrid Discrete-Continuous Diffusion Modelscited by 34
  29. 2024Cometh: A continuous-time discrete-state graph diffusion modelcited by 32
  30. 2025Attention Is All You Need for KV Cache in Diffusion LLMscited by 32
  31. 2025A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Modelscited by 30
  32. 2025WyckoffDiff -- A Generative Diffusion Model for Crystal Symmetrycited by 26
  33. 2025Target Concrete Score Matching: A Holistic Framework for Discrete Diffusioncited by 25
  34. 2024Convergence of Score-Based Discrete Diffusion Models: A Discrete-Time Analysiscited by 24
  35. 2025Training-Free Guidance Beyond Differentiability: Scalable Path Steering with Tree Search in Diffusion and Flow Modelscited by 24
  36. 2025Encoder-Decoder Diffusion Language Models for Efficient Training and Inferencecited by 24
  37. 2025Breaking AR's Sampling Bottleneck: Provable Acceleration via Diffusion Language Modelscited by 23
  38. 2025MDNS: Masked Diffusion Neural Sampler via Stochastic Optimal Controlcited by 21

50 most cited of 169 All 169 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2022How Much is Enough? A Study on Diffusion Times in Score-based Generative Models
  2. 2024Score-based Diffusion Models via Stochastic Differential Equations -- a Technical Tutorial
  3. 2021A Variational Perspective on Diffusion-Based Generative Models and Score Matching
  4. 2021Maximum Likelihood Training of Score-Based Diffusion Models
  5. 2026A Stochastic Analysis Perspective on Diffusion Models
  6. 2024From optimal score matching to optimal sampling
  7. 2026A Unified Measure-Theoretic View of Diffusion, Score-Based, and Flow Matching Generative Models
  8. 2024Discrete vs. Continuous Trade-offs for Generative Models