mlsanityData to 7 Oct 2026

Foundations of Diffusion Models in General State Spaces: A Self-Contained Introduction

arXiv 2512.05092 ↗First posted 4 Dec 2025stat.ML cs.LGcited by 1

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Although diffusion models now occupy a central place in generative modeling, introductory treatments commonly assume Euclidean data and seldom clarify their connection to discrete-state analogues. This article is a self-contained primer on diffusion over general state spaces, unifying continuous domains and discrete/categorical structures under one lens. We develop the discrete-time view (forward noising via Markov kernels and learned reverse dynamics) alongside its continuous-time limits -- stochastic differential equations (SDEs) in ℝd and continuous-time Markov chains (CTMCs) on finite alphabets -- and derive the associated Fokker--Planck and master equations. A common variational treatment yields the ELBO that underpins standard training losses. We make explicit how forward corruption choices -- Gaussian processes in continuous spaces and structured categorical transition kernels (uniform, masking/absorbing and more) in discrete spaces -- shape reverse dynamics and the ELBO. The presentation is layered for three audiences: newcomers seeking a self-contained intuitive introduction; diffusion practitioners wanting a global theoretical synthesis; and continuous-diffusion experts looking for an analogy-first path into discrete diffusion. The result is a unified roadmap to modern diffusion methodology across continuous domains and discrete sequences, highlighting a compact set of reusable proofs, identities, and core theoretical principles.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 72 of its 132 references that are in the map. First posted on arXiv 4 Dec 2025.

Builds on

72 of 132 references.

Closest ancestors · 17

Same direction, newest first

  1. 2025Feynman-Kac Correctors in Diffusion: Annealing, Guidance, and Product of Experts4 Mar
  2. 2025Debiasing Guidance for Discrete Diffusion with Sequential Monte Carlo10 Feb
  3. 2024Simple Guidance Mechanisms for Discrete Diffusion Models13 Dec
  4. 2024Unified Generation, Reconstruction, and Representation: Generalized Diffusion with Adaptive Latent Encoding-Decoding29 Feb
  5. 2024Convergence Analysis of Discrete Diffusion Model: Exact Implementation through Uniformization12 Feb
  6. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis4 Jul
  7. 2022Score-based Continuous-time Discrete Diffusion Models30 Nov
  8. 2022From Denoising Diffusions to Denoising Markov Models7 Nov
  9. 2022Categorical SDEs with Simplex Diffusion26 Oct
  10. 2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning8 Aug
  11. 2022A Continuous Time Framework for Discrete Denoising Models30 May
  12. 2022Progressive Distillation for Fast Sampling of Diffusion Models1 Feb
5 more
  1. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesis29 Nov
  2. 2021Score-based Generative Modeling in Latent Space10 Jun
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  4. 2020Denoising Diffusion Implicit Models6 Oct
  5. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 55

  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  2. 2018Neural Ordinary Differential Equationscited by 3,733
  3. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  4. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  5. 2014NICE: Non-linear Independent Components Estimationcited by 1,052
  6. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  7. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 810
  8. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  9. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
  10. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  11. 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Modelscited by 592
  12. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
38 more
  1. 2015Importance Weighted Autoencoderscited by 454
  2. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Modelscited by 439
  3. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 371
  4. 2024Discrete Flow Matchingcited by 301
  5. 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Designcited by 294
  6. 2024Flow Matching Guide and Codecited by 291
  7. 2021How to Train Your Energy-Based Modelscited by 271
  8. 2020Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Imagescited by 261
  9. 2023Flow Matching on General Geometriescited by 260
  10. 2022DiffusionBERT: Improving Generative Masked Language Models with Diffusion Modelscited by 214
  11. 2022Continuous diffusion for categorical datacited by 202
  12. 2018Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinementcited by 194
  13. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Controlcited by 191
  14. 2025Remasking Discrete Diffusion Models with Inference-Time Scalingcited by 181
  15. 2021Symbolic Music Generation with Diffusion Modelscited by 152
  16. 2023A Reparameterized Discrete Diffusion Model for Text Generationcited by 152
  17. 2022Concrete Score Matching: Generalized Score Matching for Discrete Datacited by 144
  18. 2024Unlocking Guidance for Discrete State-Space Diffusion and Flow Modelscited by 124
  19. 2025The Diffusion Dualitycited by 124
  20. 2021ImageBART: Bidirectional Context with Multinomial Diffusion for Autoregressive Image Synthesiscited by 122
  21. 2022Self-conditioned Embedding Diffusion for Text Generationcited by 114
  22. 2022Latent Diffusion for Language Generationcited by 110
  23. 2021Step-unrolled Denoising Autoencoders for Text Generationcited by 102
  24. 2024Generator Matching: Generative modeling with arbitrary Markov processescited by 82
  25. 2022Text Generation with Diffusion Language Models: A Pre-training Approach with Continuous Paragraph Denoisecited by 79
  26. 2025Generalized Interpolating Discrete Diffusioncited by 68
  27. 2023TESS: Text-to-Text Self-Conditioned Simplex Diffusioncited by 63
  28. 2025Edit Flows: Flow Matching with Edit Operationscited by 53
  29. 2025Any-Order Flexible Length Masked Diffusioncited by 48
  30. 2022DiffusER: Discrete Diffusion via Edit-based Reconstructioncited by 41
  31. 2022Empowering Diffusion Models on the Embedding Space for Text Generationcited by 40
  32. 2025Continuously Augmented Discrete Diffusion model for Categorical Generative Modelingcited by 32
  33. 2025Continuous Diffusion Model for Language Modelingcited by 29
  34. 2025Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasonercited by 28
  35. 2021Beyond In-Place Corruption: Insertion and Deletion In Denoising Probabilistic Modelscited by 23
  36. 2025Diffuse Everything: Multimodal Diffusion Models on Arbitrary State Spacescited by 19
  37. 2022Language modeling via stochastic processescited by 18
  38. 2024DiscDiff: Latent Diffusion Model for DNA Sequence Generationcited by 12

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026A Tutorial on Diffusion Theory: From Differential Equations to Diffusion Models
  2. 2023Blackout Diffusion: Generative Diffusion Models in Discrete-State Spaces
  3. 2025Non-Asymptotic Convergence of Discrete Diffusion Models: Masked and Random Walk dynamics
  4. 2024Bridging discrete and continuous state spaces: Exploring the Ehrenfest process in time-continuous diffusion models
  5. 2025Diffusion Models: A Mathematical Introduction
  6. 2025Generative diffusion models from a PDE perspective
  7. 2026Sticky Jump Diffusions: A Unifying View of Masked, Continuous, and Hybrid Diffusion
  8. 2024Demystifying Variational Diffusion Models