Foundations of Diffusion Models in General State Spaces: A Self-Contained Introduction
On the map: Diffusion models → Diffusion foundations and sampling
Abstract (arXiv)
Although diffusion models now occupy a central place in generative modeling, introductory treatments commonly assume Euclidean data and seldom clarify their connection to discrete-state analogues. This article is a self-contained primer on diffusion over general state spaces, unifying continuous domains and discrete/categorical structures under one lens. We develop the discrete-time view (forward noising via Markov kernels and learned reverse dynamics) alongside its continuous-time limits -- stochastic differential equations (SDEs) in ℝd and continuous-time Markov chains (CTMCs) on finite alphabets -- and derive the associated Fokker--Planck and master equations. A common variational treatment yields the ELBO that underpins standard training losses. We make explicit how forward corruption choices -- Gaussian processes in continuous spaces and structured categorical transition kernels (uniform, masking/absorbing and more) in discrete spaces -- shape reverse dynamics and the ELBO. The presentation is layered for three audiences: newcomers seeking a self-contained intuitive introduction; diffusion practitioners wanting a global theoretical synthesis; and continuous-diffusion experts looking for an analogy-first path into discrete diffusion. The result is a unified roadmap to modern diffusion methodology across continuous domains and discrete sequences, highlighting a compact set of reusable proofs, identities, and core theoretical principles.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 72 of its 132 references that are in the map. First posted on arXiv 4 Dec 2025.
Builds on
72 of 132 references.
Closest ancestors · 17
Same direction, newest first
- 2025Feynman-Kac Correctors in Diffusion: Annealing, Guidance, and Product of Experts4 Mar
- 2025Debiasing Guidance for Discrete Diffusion with Sequential Monte Carlo10 Feb
- 2024Simple Guidance Mechanisms for Discrete Diffusion Models13 Dec
- 2024Unified Generation, Reconstruction, and Representation: Generalized Diffusion with Adaptive Latent Encoding-Decoding29 Feb
- 2024Convergence Analysis of Discrete Diffusion Model: Exact Implementation through Uniformization12 Feb
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis4 Jul
- 2022Score-based Continuous-time Discrete Diffusion Models30 Nov
- 2022From Denoising Diffusions to Denoising Markov Models7 Nov
- 2022Categorical SDEs with Simplex Diffusion26 Oct
- 2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning8 Aug
- 2022A Continuous Time Framework for Discrete Denoising Models30 May
- 2022Progressive Distillation for Fast Sampling of Diffusion Models1 Feb
5 more
- 2021Vector Quantized Diffusion Model for Text-to-Image Synthesis29 Nov
- 2021Score-based Generative Modeling in Latent Space10 Jun
- 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
- 2020Denoising Diffusion Implicit Models6 Oct
- 2020Denoising Diffusion Probabilistic Models19 Jun
Other references · 55
- 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
- 2018Neural Ordinary Differential Equationscited by 3,733
- 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
- 2022MaskGIT: Masked Generative Image Transformercited by 1,082
- 2014NICE: Non-linear Independent Components Estimationcited by 1,052
- 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
- 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 810
- 2024Simple and Effective Masked Diffusion Language Modelscited by 800
- 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
- 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
- 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Modelscited by 592
- 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
38 more
- 2015Importance Weighted Autoencoderscited by 454
- 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Modelscited by 439
- 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 371
- 2024Discrete Flow Matchingcited by 301
- 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Designcited by 294
- 2024Flow Matching Guide and Codecited by 291
- 2021How to Train Your Energy-Based Modelscited by 271
- 2020Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Imagescited by 261
- 2023Flow Matching on General Geometriescited by 260
- 2022DiffusionBERT: Improving Generative Masked Language Models with Diffusion Modelscited by 214
- 2022Continuous diffusion for categorical datacited by 202
- 2018Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinementcited by 194
- 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Controlcited by 191
- 2025Remasking Discrete Diffusion Models with Inference-Time Scalingcited by 181
- 2021Symbolic Music Generation with Diffusion Modelscited by 152
- 2023A Reparameterized Discrete Diffusion Model for Text Generationcited by 152
- 2022Concrete Score Matching: Generalized Score Matching for Discrete Datacited by 144
- 2024Unlocking Guidance for Discrete State-Space Diffusion and Flow Modelscited by 124
- 2025The Diffusion Dualitycited by 124
- 2021ImageBART: Bidirectional Context with Multinomial Diffusion for Autoregressive Image Synthesiscited by 122
- 2022Self-conditioned Embedding Diffusion for Text Generationcited by 114
- 2022Latent Diffusion for Language Generationcited by 110
- 2021Step-unrolled Denoising Autoencoders for Text Generationcited by 102
- 2024Generator Matching: Generative modeling with arbitrary Markov processescited by 82
- 2022Text Generation with Diffusion Language Models: A Pre-training Approach with Continuous Paragraph Denoisecited by 79
- 2025Generalized Interpolating Discrete Diffusioncited by 68
- 2023TESS: Text-to-Text Self-Conditioned Simplex Diffusioncited by 63
- 2025Edit Flows: Flow Matching with Edit Operationscited by 53
- 2025Any-Order Flexible Length Masked Diffusioncited by 48
- 2022DiffusER: Discrete Diffusion via Edit-based Reconstructioncited by 41
- 2022Empowering Diffusion Models on the Embedding Space for Text Generationcited by 40
- 2025Continuously Augmented Discrete Diffusion model for Categorical Generative Modelingcited by 32
- 2025Continuous Diffusion Model for Language Modelingcited by 29
- 2025Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasonercited by 28
- 2021Beyond In-Place Corruption: Insertion and Deletion In Denoising Probabilistic Modelscited by 23
- 2025Diffuse Everything: Multimodal Diffusion Models on Arbitrary State Spacescited by 19
- 2022Language modeling via stochastic processescited by 18
- 2024DiscDiff: Latent Diffusion Model for DNA Sequence Generationcited by 12
50 most cited of 55 All 55 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.
- 2026Generalized Discrete Diffusion from Snapshotscited by 3
- 2026Generalized Discrete Diffusion from Snapshots22 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2026A Tutorial on Diffusion Theory: From Differential Equations to Diffusion Models
- 2023Blackout Diffusion: Generative Diffusion Models in Discrete-State Spaces
- 2025Non-Asymptotic Convergence of Discrete Diffusion Models: Masked and Random Walk dynamics
- 2024Bridging discrete and continuous state spaces: Exploring the Ehrenfest process in time-continuous diffusion models
- 2025Diffusion Models: A Mathematical Introduction
- 2025Generative diffusion models from a PDE perspective
- 2026Sticky Jump Diffusions: A Unifying View of Masked, Continuous, and Hybrid Diffusion
- 2024Demystifying Variational Diffusion Models