# The Effect of Stochasticity in Score-Based Diffusion Sampling: a KL Divergence Analysis arXiv: https://arxiv.org/abs/2506.11378 ยท HTML: https://mlsanity.com/paper/2506.11378/ ## Key facts - Cited by 1 papers in the map (data through 14 Oct 2026). - Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. - Builds on 31 of its 65 references that are in the map. - First posted on arXiv 13 Jun 2025. ## Placed in - Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/) ## Builds on (most cited first, 30 of 31) - [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020) - [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020) - [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022) - [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022) - [Stochastic Interpolants: A Unifying Framework for Flows and Diffusions](https://mlsanity.com/paper/2303.08797/) (2023) - [SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers](https://mlsanity.com/paper/2401.08740/) (2024) - [Maximum Likelihood Training of Score-Based Diffusion Models](https://mlsanity.com/paper/2101.09258/) (2021) - [Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions](https://mlsanity.com/paper/2209.11215/) (2022) - [Assessing Generative Models via Precision and Recall](https://mlsanity.com/paper/1806.00035/) (2018) - [Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.06503/) (2022) - [Diffusion Models for Medical Image Analysis: A Comprehensive Survey](https://mlsanity.com/paper/2211.07804/) (2022) - [Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localization](https://mlsanity.com/paper/2308.03686/) (2023) - [Gotta Go Fast When Generating Data with Score-Based Models](https://mlsanity.com/paper/2105.14080/) (2021) - [Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptions](https://mlsanity.com/paper/2211.01916/) (2022) - [VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learning](https://mlsanity.com/paper/1705.07761/) (2017) - [Convergence for score-based generative modeling with polynomial complexity](https://mlsanity.com/paper/2206.06227/) (2022) - [GenCast: Diffusion-based ensemble forecasting for medium-range weather](https://mlsanity.com/paper/2312.15796/) (2023) - [Understanding Hallucinations in Diffusion Models through Mode Interpolation](https://mlsanity.com/paper/2406.09358/) (2024) - [Restart Sampling for Improving Generative Processes](https://mlsanity.com/paper/2306.14878/) (2023) - [SA-Solver: Stochastic Adams Solver for Fast Sampling of Diffusion Models](https://mlsanity.com/paper/2309.05019/) (2023) - [A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AI](https://mlsanity.com/paper/2303.13336/) (2023) - [KL Convergence Guarantees for Score diffusion models under minimal data assumptions](https://mlsanity.com/paper/2308.12240/) (2023) - [The Blessing of Randomness: SDE Beats ODE in General Diffusion-based Image Editing](https://mlsanity.com/paper/2311.01410/) (2023) - [Probability flow solution of the Fokker-Planck equation](https://mlsanity.com/paper/2206.04642/) (2022) - [Feature Likelihood Divergence: Evaluating the Generalization of Generative Models Using Samples](https://mlsanity.com/paper/2302.04440/) (2023) - [Empowering Time Series Analysis with Foundation Models: A Comprehensive Survey](https://mlsanity.com/paper/2405.02358/) (2024) - [A Survey of Diffusion Models in Natural Language Processing](https://mlsanity.com/paper/2305.14671/) (2023) - [Exploring the Optimal Choice for Generative Processes in Diffusion Models: Ordinary vs Stochastic Differential Equations](https://mlsanity.com/paper/2306.02063/) (2023) - [Closing the ODE-SDE gap in score-based diffusion models through the Fokker-Planck equation](https://mlsanity.com/paper/2311.15996/) (2023) - [Controlled Latent Diffusion Models for 3D Porous Media Reconstruction](https://mlsanity.com/paper/2503.24083/) (2025) ## Built on by (most cited first, 1 of 1) - [A Unified Kullback--Leibler Divergence Analysis of Generative Diffusion Models via Entropy Production Rate](https://mlsanity.com/paper/2608.02406/) (2026) ## Abstract (arXiv) Sampling in score-based diffusion models can be performed by solving either a reverse-time stochastic differential equation (SDE) parameterized by an arbitrary stochasticity function or a probability flow ODE, corresponding to setting this stochasticity function to zero. In this work, we investigate the effect of this stochasticity on the generation process through the evolution of Kullback-Leibler (KL) divergences, obtaining general KL divergence bounds and a novel analysis of the impact of the time-profile of the score error on model performance. For exact score functions, stochasticity has a contractive effect, decreasing KL divergence along the sampling trajectory. For approximate scores, however, a trade-off arises between correcting accumulated errors and amplifying current score errors, meaning stochasticity can either improve or degrade generation performance. Theoretical considerations indicate that the gain from stochasticity depends on the time-localization of the trained model error. We test this in experiments on both toy and benchmark data sets, also comparing the KL divergence evolution with the obtained bounds. We also present a fully analytical example, where all the relevant quantities can be computed, and the optimal stochasticity function can be characterized via an optimal control analysis.