mlsanityData to 7 Oct 2026

How Discrete and Continuous Diffusion Meet: Comprehensive Analysis of Discrete Diffusion Models via a Stochastic Integral Framework

arXiv 2410.03601 ↗First posted 4 Oct 2024cs.LG cs.NA math.NAcited by 48

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Discrete diffusion models have gained increasing attention for their ability to model complex distributions with tractable sampling and inference. However, the error analysis for discrete diffusion models remains less well-understood. In this work, we propose a comprehensive framework for the error analysis of discrete diffusion models based on L'evy-type stochastic integrals. By generalizing the Poisson random measure to that with a time-independent and state-dependent intensity, we rigorously establish a stochastic integral formulation of discrete diffusion models and provide the corresponding change of measure theorems that are intriguingly analogous to Ito integrals and Girsanov's theorem for their continuous counterparts. Our framework unifies and strengthens the current theoretical results on discrete diffusion models and obtains the first error bound for the τ-leaping scheme in KL divergence. With error sources clearly identified, our analysis gives new insight into the mathematical properties of discrete diffusion models and offers guidance for the design of efficient and accurate algorithms for real-world discrete diffusion model applications.

Key facts. Cited by 48 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 90 of its 130 references that are in the map. First posted on arXiv 4 Oct 2024.

Builds on

90 of 130 references.

Closest ancestors · 41

Same direction, newest first

  1. 2024Convergence of Score-Based Discrete Diffusion Models: A Discrete-Time Analysis3 Oct
  2. 2024Derivative-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value-Based Decoding15 Aug
  3. 2024A Sharp Convergence Theory for The Probability Flow ODEs of Diffusion Models5 Aug
  4. 2024Discrete generative diffusion models without stochastic differential equations: a tensor network approach15 Jul
  5. 2024Accelerating Diffusion Models with Parallel Sampling: Inference at Sub-Linear Time Complexity24 May
  6. 2024Bridging discrete and continuous state spaces: Exploring the Ehrenfest process in time-continuous diffusion models6 May
  7. 2024Convergence Analysis of Probability Flow ODE for Score-based Generative Models15 Apr
  8. 2024An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimization11 Apr
  9. 2024Quantum State Generation with Structure-Preserving Diffusion Model9 Apr
  10. 2024Tutorial on Diffusion Models for Imaging and Vision26 Mar
  11. 2024Accelerating Convergence of Score-Based Diffusion Models, Provably6 Mar
  12. 2024Accelerating Parallel Sampling of Diffusion Models15 Feb
29 more
  1. 2024Convergence Analysis of Discrete Diffusion Model: Exact Implementation through Uniformization12 Feb
  2. 2024Convergence Analysis for General Probability Flow ODEs of Diffusion Models in Wasserstein Distances31 Jan
  3. 2023A Note on the Convergence of Denoising Diffusion Probabilistic Models10 Dec
  4. 2023Generative Diffusion Models for Lattice Field Theory6 Nov
  5. 2023Diffusion on the Probability Simplex5 Sep
  6. 2023Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localization7 Aug
  7. 2023Towards Faster Non-Asymptotic Convergence for Diffusion-Based Generative Models15 Jun
  8. 2023Error Bounds for Flow Matching Methods26 May
  9. 2023Parallel Sampling of Diffusion Models25 May
  10. 2023Improved Convergence of Score-Based Diffusion Models via Prediction-Correction23 May
  11. 2023The probability flow ODE is provably fast19 May
  12. 2023Blackout Diffusion: Generative Diffusion Models in Discrete-State Spaces18 May
  13. 2023Restoration-Degradation Beyond Linear Diffusions: A Non-Asymptotic Analysis For DDIM-Type Samplers6 Mar
  14. 2022Score-based Continuous-time Discrete Diffusion Models30 Nov
  15. 2022From Denoising Diffusions to Denoising Markov Models7 Nov
  16. 2022Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptions3 Nov
  17. 2022Categorical SDEs with Simplex Diffusion26 Oct
  18. 2022Convergence of score-based generative modeling for general data distributions26 Sep
  19. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions22 Sep
  20. 2022Diffusion Models: A Comprehensive Survey of Methods and Applications2 Sep
  21. 2022Convergence for score-based generative modeling with polynomial complexity13 Jun
  22. 2022A Continuous Time Framework for Discrete Denoising Models30 May
  23. 2022Fast Sampling of Diffusion Models with Exponential Integrator29 Apr
  24. 2021Global Context with Discrete Diffusion in Vector Quantised Modelling for Image Generation3 Dec
  25. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesis29 Nov
  26. 2021Structured Denoising Diffusion Models in Discrete State-Spaces7 Jul
  27. 2021Maximum Likelihood Training of Score-Based Diffusion Models22 Jan
  28. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  29. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 49

  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  2. 2022Flow Matching for Generative Modelingcited by 5,426
  3. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  4. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  5. 2022Building Normalizing Flows with Stochastic Interpolantscited by 964
  6. 2023Stochastic Interpolants: A Unifying Framework for Flows and Diffusionscited by 897
  7. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  8. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
  9. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  10. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  11. 2022GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generationcited by 511
  12. 2022DiGress: Discrete Denoising diffusion for graph generationcited by 503
37 more
  1. 2020GraphAF: a Flow-based Autoregressive Model for Molecular Graph Generationcited by 301
  2. 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Designcited by 294
  3. 2020Permutation Invariant Graph Generation via Score-Based Generative Modelingcited by 263
  4. 2022Diffsound: Discrete Diffusion Model for Text-to-sound Generationcited by 256
  5. 2022DiffusionBERT: Improving Generative Masked Language Models with Diffusion Modelscited by 214
  6. 2021Autoregressive Diffusion Modelscited by 208
  7. 2023Protein Design with Guided Discrete Diffusioncited by 154
  8. 2023A Reparameterized Discrete Diffusion Model for Text Generationcited by 152
  9. 2022Concrete Score Matching: Generalized Score Matching for Discrete Datacited by 144
  10. 2023LayoutDM: Discrete Diffusion Model for Controllable Layout Generationcited by 141
  11. 2024Unlocking Guidance for Discrete State-Space Diffusion and Flow Modelscited by 124
  12. 2020Generative Modeling with Denoising Auto-Encoders and Langevin Samplingcited by 118
  13. 2022Parallel Diffusion Models of Operator and Image for Blind Inverse Problemscited by 113
  14. 2023AR-Diffusion: Auto-Regressive Diffusion Model for Text Generationcited by 113
  15. 2024Dirichlet Flow Matching with Applications to DNA Sequence Designcited by 110
  16. 2019Theoretical guarantees for sampling and inference in generative models with latent diffusionscited by 97
  17. 2023Dirichlet Diffusion Score Model for Biological Sequence Generationcited by 73
  18. 2024Fisher Flow Matching for Generative Modeling over Discrete Datacited by 68
  19. 2022Discrete Contrastive Diffusion for Cross-Modal Music and Image Generationcited by 44
  20. 2023Graph Denoising Diffusion for Inverse Protein Foldingcited by 44
  21. 2023LayoutDiffusion: Improving Graphic Layout Generation by Discrete Diffusion Probabilistic Modelscited by 40
  22. 2023Protein Discovery with Discrete Walk-Jump Samplingcited by 38
  23. 2023DiverseMotion: Towards Diverse Human Motion Generation via Discrete Diffusioncited by 36
  24. 2023DiffuSeq-v2: Bridging Discrete and Continuous Text Spaces for Accelerated Seq2Seq Diffusion Modelscited by 36
  25. 2023RetroBridge: Modeling Retrosynthesis with Markov Bridgescited by 35
  26. 2018Monge-Amp`ere Flow for Generative Modelingcited by 33
  27. 2023Sparse Training of Discrete Diffusion Models for Graph Generationcited by 26
  28. 2024Diffusion Models in textitDe Novo Drug Designcited by 26
  29. 2024M2D2M: Multi-Motion Generation from Text with Discrete Diffusion Modelscited by 23
  30. 2023Diffusion-NAT: Self-Prompting Discrete Diffusion for Non-Autoregressive Text Generationcited by 21
  31. 2022Plug & Play Directed Evolution of Proteins with Gradient-based Discrete MCMCcited by 17
  32. 2023Deep Equilibrium Diffusion Restoration with Parallel Samplingcited by 17
  33. 2019Discrete Object Generation with Reversible Inductive Constructioncited by 16
  34. 2023DCTTS: Discrete Diffusion Model with Contrastive Learning for Text-to-speech Generationcited by 9
  35. 2024Training-Free Guidance for Discrete Diffusion Models for Molecular Generationcited by 6
  36. 2023Fast non-autoregressive inverse folding with discrete diffusioncited by 5
  37. 2024Improving Discrete Diffusion Models via Structured Preferential Generationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 28; 2026: 20.

48 papers.

Sort:

By area:

  1. 2025Inference-Time Alignment in Diffusion Models with Reward-Guided Generation: Tutorial and Reviewcited by 82
  2. 2025Path Planning for Masked Diffusion Model Samplingcited by 79
  3. 2025Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithmscited by 47
  4. 2025KLASS: KL-Guided Fast Inference in Masked Diffusion Modelscited by 44
  5. 2025DriftLite: Lightweight Drift Control for Inference-Time Scaling of Diffusion Modelscited by 21
  6. 2025Absorb and Converge: Provable Convergence Guarantee for Absorbing Discrete Diffusion Modelscited by 20
  7. 2025MDNS: Masked Diffusion Neural Sampler via Stochastic Optimal Controlcited by 20
  8. 2025Discrete Diffusion Models: Novel Analysis and New Sampler Guaranteescited by 20
  9. 2025Diffuse Everything: Multimodal Diffusion Models on Arbitrary State Spacescited by 19
  10. 2025Solving Inverse Problems via Diffusion-Based Priors: An Approximation-Free Ensemble Sampling Approachcited by 18
  11. 2025DiFFPO: Training Diffusion LLMs to Reason Fast and Furious via Reinforcement Learningcited by 17
  12. 2026Confidence-Based Decoding is Provably Efficient for Diffusion Language Modelscited by 17
36 more
  1. 2025Proximal Diffusion Neural Samplercited by 16
  2. 2025Enhancing Reasoning for Diffusion LLMs via Distribution Matching Policy Optimizationcited by 15
  3. 2025Planner Aware Path Learning in Diffusion Language Models Trainingcited by 14
  4. 2025Complexity Analysis of Normalizing Constant Estimation: from Jarzynski Equality to Annealed Importance Sampling and beyondcited by 11
  5. 2025Flexible-length Text Infilling for Discrete Diffusion Modelscited by 11
  6. 2025Non-Asymptotic Convergence of Discrete Diffusion Models: Masked and Random Walk dynamicscited by 11
  7. 2025Demystifying MaskGIT Sampler and Beyond: Adaptive Order Selection in Masked Diffusioncited by 10
  8. 2025Error Analysis of Discrete Flow with Generator Matchingcited by 9
  9. 2026Discrete Feynman-Kac Correctorscited by 9
  10. 2026Adaptation to Intrinsic Dependence in Diffusion Language Modelscited by 9
  11. 2025Learnable Sampler Distillation for Discrete Diffusion Modelscited by 8
  12. 2026Sharp Convergence Rates for Masked Diffusion Modelscited by 8
  13. 2026The data geometry of masking diffusion: Certified-optimal schedules via unmasking growth complexitycited by 8
  14. 2025Bit-Level Discrete Diffusion with Markov Probabilistic Models: An Improved Framework with Sharp Convergence Bounds under Minimal Assumptionscited by 4
  15. 2025Discrete State Diffusion Models: A Sample Complexity Perspectivecited by 4
  16. 2026Discrete Adjoint Schr"odinger Bridge Samplercited by 4
  17. 2026Corrected Samplers for Discrete Flow Modelscited by 3
  18. 2025INFO-SEDD: Continuous Time Markov Chains as Scalable Information Metrics Estimatorscited by 2
  19. 2025On Efficiency-Effectiveness Trade-off of Diffusion-based Recommenderscited by 2
  20. 2026Simple Approximation and Derivative Free Inference-Time Scaling for Diffusion Models via Sequential Monte Carlo on Path Measurescited by 2
  21. 2026The information geometry of product-reference discrete diffusion: Interaction growth complexity and optimal schedulingcited by 2
  22. 2025Discrete Spatial Diffusion: Intensity-Preserving Diffusion Modelingcited by 1
  23. 2025Multimodal Generative Flows for LHC Jetscited by 1
  24. 2025Deterministic Discrete Denoisingcited by 1
  25. 2026From Scores to Gibbs Correctors: Accelerating Uniform-Rate Discrete Diffusion Modelscited by 1
  26. 2026Diffusion Models for Adaptive Sequential Data Generationcited by 1
  27. 2026Accelerating Discrete Diffusion Models with Parallel-In-Time Samplingcited by 1
  28. 2026Multi-Mask Diffusion Language Models for Few-Step Generationcited by 1
  29. 2026Minimax Optimality of Score-Entropy Discrete Diffusioncited by 1
  30. 2026FluxLite: Inference-Time Proposal Control for Discrete Diffusion Modelscited by 1
  31. 2025On the ε-Free Inference Complexity of Absorbing Discrete Diffusioncited by 0
  32. 2026Discrete Diffusion with Sample-Efficient Estimators for Conditionalscited by 0
  33. 2026Binomial flows: Denoising and flow matching for discrete ordinal datacited by 0
  34. 2026On the Error-Correcting Effects of Stochasticity in Discrete Diffusioncited by 0
  35. 2026A Time-Reparameterized Cumulative Intensity Extrapolation Sampler for Discrete Flow Matchingcited by 0
  36. 2026Sample complexity bounds for categorical Markov random fields via Discrete Diffusionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Efficient Sampling with Discrete Diffusion Models: Sharp and Adaptive Guarantees
  2. 2025Foundations of Diffusion Models in General State Spaces: A Self-Contained Introduction
  3. 2026A Mathematical Introduction to Diffusion Models
  4. 2023A Geometric Perspective on Diffusion Models
  5. 2024Non-asymptotic bounds for forward processes in denoising diffusions: Ornstein-Uhlenbeck is hard to beat
  6. 2025Diffusion Models under Alternative Noise: Simplified Analysis and Sensitivity
  7. 2026Sticky Jump Diffusions: A Unifying View of Masked, Continuous, and Hybrid Diffusion
  8. 2025Information-Theoretic Proofs for Diffusion Sampling