A Unified Kullback--Leibler Divergence Analysis of Generative Diffusion Models via Entropy Production Rate
On the map: Diffusion models → Diffusion foundations and sampling
Abstract (arXiv)
We introduce a unified framework for the error analysis of generative models based on the entropy production rate of the forward-reverse diffusion process pair. For a pair of continuity equation flows, the rate admits a closed velocity form identity whose time integral decomposes the terminal Kullback--Leibler (KL) divergence into the sum of an initialization error, a score approximation error, and a time-discretization error. By analyzing the entropy production at the level of marginal distributions, rather than in path space, our framework yields a sharp convergence rate of O(h2) for the Euler-Maruyama sampler, where h is the step size. This improves upon the O(h) rates typically obtained from Girsanov's path-space analyses. Furthermore, our framework unifies the analysis of score-based SDEs, probability-flow ODEs, and stochastic interpolants by varying diffusion coefficients within a single inequality, revealing the trade-off between deterministic and stochastic sampling. Numerical experiments confirm the predicted scaling with step size and terminal time.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 23 of its 54 references that are in the map. First posted on arXiv 3 Aug 2026.
Builds on
23 of 54 references.
Closest ancestors · 17
Same direction, newest first
- 2025The Effect of Stochasticity in Score-Based Diffusion Sampling: a KL Divergence Analysis13 Jun
- 2024Neural Entropy5 Sep
- 2024Diffusion models for Gaussian distributions: Exact solutions and Wasserstein errors23 May
- 2024Convergence Analysis for General Probability Flow ODEs of Diffusion Models in Wasserstein Distances31 Jan
- 2023Generative Diffusion From An Action Principle6 Oct
- 2023KL Convergence Guarantees for Score diffusion models under minimal data assumptions23 Aug
- 2023Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localization7 Aug
- 2023The probability flow ODE is provably fast19 May
- 2023Fast Diffusion Probabilistic Model Sampling through the lens of Backward Error Analysis22 Apr
- 2022Convergence of score-based generative modeling for general data distributions26 Sep
- 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions22 Sep
- 2022Convergence of denoising diffusion models under the manifold hypothesis10 Aug
5 more
- 2022Convergence for score-based generative modeling with polynomial complexity13 Jun
- 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
- 2021Variational Diffusion Models1 Jul
- 2021Improved Denoising Diffusion Probabilistic Models18 Feb
- 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
Other references · 6
- 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
- 2023Stochastic Interpolants: A Unifying Framework for Flows and Diffusionscited by 897
- 2023Convergence of flow-based generative models via proximal gradient descent in Wasserstein spacecited by 34
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026A Gaussian Perspective for Distributional Discrepancy in Generative Diffusion Models
- 2025A Sharp KL-Convergence Analysis for Diffusion Models under Minimal Assumptions
- 2026Entropy-Based Dimension-Free Convergence and Loss-Adaptive Schedules for Diffusion Models
- 2024A solvable generative model with a linear, one-step denoiser
- 2023Towards Faster Non-Asymptotic Convergence for Diffusion-Based Generative Models
- 2026A Tutorial on Diffusion Theory: From Differential Equations to Diffusion Models
- 2025A PDE Perspective on Generative Diffusion Models
- 2024A Sharp Convergence Theory for The Probability Flow ODEs of Diffusion Models