# Closing the ODE-SDE gap in score-based diffusion models through the Fokker-Planck equation arXiv: https://arxiv.org/abs/2311.15996 ยท HTML: https://mlsanity.com/paper/2311.15996/ ## Key facts - Cited by 12 papers in the map (data through 14 Oct 2026). - Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. - Builds on 18 of its 27 references that are in the map. - First posted on arXiv 27 Nov 2023. ## Placed in - Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/) ## Builds on (most cited first, 18 of 18) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020) - [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021) - [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015) - [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022) - [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019) - [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021) - [Maximum Likelihood Training of Score-Based Diffusion Models](https://mlsanity.com/paper/2101.09258/) (2021) - [Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions](https://mlsanity.com/paper/2209.11215/) (2022) - [Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localization](https://mlsanity.com/paper/2308.03686/) (2023) - [Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptions](https://mlsanity.com/paper/2211.01916/) (2022) - [Convergence for score-based generative modeling with polynomial complexity](https://mlsanity.com/paper/2206.06227/) (2022) - [The probability flow ODE is provably fast](https://mlsanity.com/paper/2305.11798/) (2023) - [Error Bounds for Flow Matching Methods](https://mlsanity.com/paper/2305.16860/) (2023) - [Maximum Likelihood Training for Score-Based Diffusion ODEs by High-Order Denoising Score Matching](https://mlsanity.com/paper/2206.08265/) (2022) - [FP-Diffusion: Improving Score-based Diffusion Models by Enforcing the Underlying Score Fokker-Planck Equation](https://mlsanity.com/paper/2210.04296/) (2022) - [Improved Convergence of Score-Based Diffusion Models via Prediction-Correction](https://mlsanity.com/paper/2305.14164/) (2023) ## Built on by (most cited first, 12 of 12) - [Accelerating Diffusion Models with Parallel Sampling: Inference at Sub-Linear Time Complexity](https://mlsanity.com/paper/2405.15986/) (2024) - [Unifying Generation and Prediction on Graphs with Latent Graph Diffusion](https://mlsanity.com/paper/2402.02518/) (2024) - [Rethinking Diffusion Posterior Sampling: From Conditional Score Estimator to Maximizing a Posterior](https://mlsanity.com/paper/2501.18913/) (2025) - [A Survey on Diffusion Models for Anomaly Detection](https://mlsanity.com/paper/2501.11430/) (2025) - [Diffusion-PINN Sampler](https://mlsanity.com/paper/2410.15336/) (2024) - [Sublinear iterations can suffice even for DDPMs](https://mlsanity.com/paper/2511.04844/) (2025) - [HJ-sampler: A Bayesian sampler for inverse problems of a stochastic process by leveraging Hamilton-Jacobi PDEs and score-based generative models](https://mlsanity.com/paper/2409.09614/) (2024) - [The Effect of Stochasticity in Score-Based Diffusion Sampling: a KL Divergence Analysis](https://mlsanity.com/paper/2506.11378/) (2025) - [Optimizing Visual Generative Models via Distribution-wise Rewards](https://mlsanity.com/paper/2607.02291/) (2026) - [A Theory of Speciation in Generative Diffusion Models on Compact Riemannian Manifolds](https://mlsanity.com/paper/2608.23798/) (2026) - [Fast Training-free Perceptual Image Compression](https://mlsanity.com/paper/2506.16102/) (2025) - [A Plug-in Interpretation of Conditioning in Score-Based Diffusion Models](https://mlsanity.com/paper/2608.19504/) (2026) ## Abstract (arXiv) Score-based diffusion models have emerged as one of the most promising frameworks for deep generative modelling, due to their state-of-the art performance in many generation tasks while relying on mathematical foundations such as stochastic differential equations (SDEs) and ordinary differential equations (ODEs). Empirically, it has been reported that ODE based samples are inferior to SDE based samples. In this paper we rigorously describe the range of dynamics and approximations that arise when training score-based diffusion models, including the true SDE dynamics, the neural approximations, the various approximate particle dynamics that result, as well as their associated Fokker--Planck equations and the neural network approximations of these Fokker--Planck equations. We systematically analyse the difference between the ODE and SDE dynamics of score-based diffusion models, and link it to an associated Fokker--Planck equation. We derive a theoretical upper bound on the Wasserstein 2-distance between the ODE- and SDE-induced distributions in terms of a Fokker--Planck residual. We also show numerically that conventional score-based diffusion models can exhibit significant differences between ODE- and SDE-induced distributions which we demonstrate using explicit comparisons. Moreover, we show numerically that reducing the Fokker--Planck residual by adding it as an additional regularisation term leads to closing the gap between ODE- and SDE-induced distributions. Our experiments suggest that this regularisation can improve the distribution generated by the ODE, however that this can come at the cost of degraded SDE sample quality.