mlsanityData to 7 Oct 2026

Categorical SDEs with Simplex Diffusion

arXiv 2210.14784 ↗First posted 26 Oct 2022cs.LGcited by 42

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models typically operate in the standard framework of generative modelling by producing continuously-valued datapoints. To this end, they rely on a progressive Gaussian smoothing of the original data distribution, which admits an SDE interpretation involving increments of a standard Brownian motion. However, some applications such as text generation or reinforcement learning might naturally be better served by diffusing categorical-valued data, i.e., lifting the diffusion to a space of probability distributions. To this end, this short theoretical note proposes Simplex Diffusion, a means to directly diffuse datapoints located on an n-dimensional probability simplex. We show how this relates to the Dirichlet distribution on the simplex and how the analogous SDE is realized thanks to a multi-dimensional Cox-Ingersoll-Ross process (abbreviated as CIR), previously used in economics and mathematical finance. Finally, we make remarks as to the numerical implementation of trajectories of the CIR process, and discuss some limitations of our approach.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 23 of its 40 references that are in the map. First posted on arXiv 26 Oct 2022.

Builds on

23 of 40 references.

Closest ancestors · 5

Same direction, newest first

  1. 2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning8 Aug
  2. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  3. 2021Maximum Likelihood Training of Score-Based Diffusion Models22 Jan
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  5. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 18

  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  2. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  4. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  5. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  6. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  7. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  8. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  9. 2022Equivariant Diffusion for Molecule Generation in 3Dcited by 666
  10. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
  11. 2022DiffDock: Diffusion Steps, Twists, and Turns for Molecular Dockingcited by 307
  12. 2021Diff-TTS: A Denoising Diffusion Model for Text-to-Speechcited by 156
6 more
  1. 2022ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speechcited by 147
  2. 2015A Complete Recipe for Stochastic Gradient MCMCcited by 146
  3. 2021Step-unrolled Denoising Autoencoders for Text Generationcited by 102
  4. 2022Language modeling via stochastic processescited by 18
  5. 2017On Wasserstein Reinforcement Learning and the Fokker-Planck equationcited by 9
  6. 2018Large-Scale Stochastic Sampling from the Probability Simplexcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 7; 2024: 14; 2025: 10; 2026: 10.

42 papers.

Sort:

By area:

  1. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  2. 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Designcited by 294
  3. 2022Continuous diffusion for categorical datacited by 202
  4. 2023A Reparameterized Discrete Diffusion Model for Text Generationcited by 152
  5. 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 145
  6. 2024Unlocking Guidance for Discrete State-Space Diffusion and Flow Modelscited by 124
  7. 2024Dirichlet Flow Matching with Applications to DNA Sequence Designcited by 110
  8. 2023Reflected Diffusion Modelscited by 81
  9. 2024Variational Flow Matching for Graph Generationcited by 78
  10. 2023Dirichlet Diffusion Score Model for Biological Sequence Generationcited by 73
  11. 2023TESS: Text-to-Text Self-Conditioned Simplex Diffusioncited by 63
  12. 2023Bayesian Flow Networkscited by 62
30 more
  1. 2024Score-based Diffusion Models via Stochastic Differential Equations -- a Technical Tutorialcited by 60
  2. 2024Mixed Continuous and Categorical Flow Matching for 3D De Novo Molecule Generationcited by 53
  3. 2024How Discrete and Continuous Diffusion Meet: Comprehensive Analysis of Discrete Diffusion Models via a Stochastic Integral Frameworkcited by 48
  4. 2025Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithmscited by 47
  5. 2024Accelerating Diffusion Models with Parallel Sampling: Inference at Sub-Linear Time Complexitycited by 42
  6. 2023Mirror Diffusion Models for Constrained and Watermarked Generationcited by 38
  7. 2024Full-Atom Peptide Design based on Multi-modal Flow Matchingcited by 34
  8. 2024Contractive Diffusion Probabilistic Modelscited by 29
  9. 2024Unifying Bayesian Flow Networks and Diffusion Models through Stochastic Differential Equationscited by 28
  10. 2025Solving Inverse Problems via Diffusion-Based Priors: An Approximation-Free Ensemble Sampling Approachcited by 18
  11. 2023Diffusion on the Probability Simplexcited by 14
  12. 2025LLaDA-MedV: Exploring Large Language Diffusion Models for Biomedical Image Understandingcited by 12
  13. 2026Continuous Latent Diffusion Language Modelcited by 12
  14. 2025Non-Asymptotic Convergence of Discrete Diffusion Models: Masked and Random Walk dynamicscited by 11
  15. 2024Hotspot-Driven Peptide Design via Multi-Fragment Autoregressive Extensioncited by 9
  16. 2026Membership Inference Attacks Against Fine-tuned Diffusion Language Modelscited by 5
  17. 2025TFG-Flow: Training-free Guidance in Multimodal Generative Flowcited by 4
  18. 2025Bit-Level Discrete Diffusion with Markov Probabilistic Models: An Improved Framework with Sharp Convergence Bounds under Minimal Assumptionscited by 4
  19. 2026Adjoint Matching through the Lens of the Stochastic Maximum Principle in Optimal Controlcited by 4
  20. 2025A diffusion-based generative model for financial time series via geometric Brownian motioncited by 2
  21. 2026Unrestrained Simplex Denoising for Discrete Data. A Non-Markovian Approach Applied to Graph Generationcited by 2
  22. 2026Simplex Relaxation for Discrete Diffusioncited by 2
  23. 2024Synthetic location trajectory generation using categorical diffusion modelscited by 1
  24. 2025Foundations of Diffusion Models in General State Spaces: A Self-Contained Introductioncited by 1
  25. 2025A Unification of Discrete, Gaussian, and Simplicial Diffusioncited by 1
  26. 2026Efficient Long-Context Modeling in Diffusion Language Models via Block Approximate Sparse Attentioncited by 1
  27. 2026A Continuous-Time Reinforcement Learning Framework for Fine-Tuning Discrete Diffusion Modelscited by 1
  28. 2026Simplex Diffusion Modelscited by 1
  29. 2026Tweedie's Formulae and Diffusion Generative Models Beyond Gaussiancited by 0
  30. 2026CARD: Calibration via Agreement in Reverse Diffusion for Out-of-Domain MRI Segmentationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Generative diffusion models from a PDE perspective
  2. 2025Beyond Linear Diffusions: Improved Representations for Rare Conditional Generative Modeling
  3. 2023Diffusion Models for Generative Artificial Intelligence: An Introduction for Applied Mathematicians
  4. 2026A Tutorial on Diffusion Theory: From Differential Equations to Diffusion Models
  5. 2022A Flexible Diffusion Model
  6. 2023Towards Faster Non-Asymptotic Convergence for Diffusion-Based Generative Models
  7. 2024Unified Discrete Diffusion for Categorical Data
  8. 2025A Sharp KL-Convergence Analysis for Diffusion Models under Minimal Assumptions