# Exploring the Optimal Choice for Generative Processes in Diffusion Models: Ordinary vs Stochastic Differential Equations arXiv: https://arxiv.org/abs/2306.02063 ยท HTML: https://mlsanity.com/paper/2306.02063/ ## Key facts - Cited by 15 papers in the map (data through 14 Oct 2026). - Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. - Builds on 23 of its 39 references that are in the map. - First posted on arXiv 3 Jun 2023. ## Placed in - Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/) ## Builds on (most cited first, 23 of 23) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020) - [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020) - [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015) - [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019) - [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018) - [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022) - [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023) - [Diffusion Models in Vision: A Survey](https://mlsanity.com/paper/2209.04747/) (2022) - [Stochastic Interpolants: A Unifying Framework for Flows and Diffusions](https://mlsanity.com/paper/2303.08797/) (2023) - [Fast Sampling of Diffusion Models with Exponential Integrator](https://mlsanity.com/paper/2204.13902/) (2022) - [Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions](https://mlsanity.com/paper/2209.11215/) (2022) - [Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.06503/) (2022) - [Perception Prioritized Training of Diffusion Models](https://mlsanity.com/paper/2204.00227/) (2022) - [Convergence of denoising diffusion models under the manifold hypothesis](https://mlsanity.com/paper/2208.05314/) (2022) - [Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptions](https://mlsanity.com/paper/2211.01916/) (2022) - [A Variational Perspective on Diffusion-Based Generative Models and Score Matching](https://mlsanity.com/paper/2106.02808/) (2021) - [Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data](https://mlsanity.com/paper/2302.07194/) (2023) - [gDDIM: Generalized denoising diffusion implicit models](https://mlsanity.com/paper/2206.05564/) (2022) - [Convergence of Langevin MCMC in KL-divergence](https://mlsanity.com/paper/1705.09048/) (2017) - [Amortised MAP Inference for Image Super-resolution](https://mlsanity.com/paper/1610.04490/) (2016) - [On sampling from a log-concave density using kinetic Langevin diffusions](https://mlsanity.com/paper/1807.09382/) (2018) - [FP-Diffusion: Improving Score-based Diffusion Models by Enforcing the Underlying Score Fokker-Planck Equation](https://mlsanity.com/paper/2210.04296/) (2022) ## Built on by (most cited first, 15 of 15) - [Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Design](https://mlsanity.com/paper/2402.04997/) (2024) - [On the Design Fundamentals of Diffusion Models: A Survey](https://mlsanity.com/paper/2306.04542/) (2023) - [Accelerating Diffusion Models with Parallel Sampling: Inference at Sub-Linear Time Complexity](https://mlsanity.com/paper/2405.15986/) (2024) - [Unifying Generation and Prediction on Graphs with Latent Graph Diffusion](https://mlsanity.com/paper/2402.02518/) (2024) - [Eta Inversion: Designing an Optimal Eta Function for Diffusion-based Real Image Editing](https://mlsanity.com/paper/2403.09468/) (2024) - [FLEX: A Backbone for Diffusion-Based Modeling of Spatio-temporal Physical Systems](https://mlsanity.com/paper/2505.17351/) (2025) - [Recent Advances in Hypergraph Neural Networks](https://mlsanity.com/paper/2503.07959/) (2025) - [Sublinear iterations can suffice even for DDPMs](https://mlsanity.com/paper/2511.04844/) (2025) - [Deep Neural Networks Inspired by Differential Equations](https://mlsanity.com/paper/2510.09685/) (2025) - [QonFusion -- Quantum Approaches to Gaussian Random Variables: Applications in Stable Diffusion and Brownian Motion](https://mlsanity.com/paper/2309.16258/) (2023) - [Non-equilibrium active noise enhances generative memory in diffusion models](https://mlsanity.com/paper/2411.07233/) (2024) - [The Effect of Stochasticity in Score-Based Diffusion Sampling: a KL Divergence Analysis](https://mlsanity.com/paper/2506.11378/) (2025) - [Improving Diffusion Generative Models via Truncated Karhunen--Lo`eve Expansion](https://mlsanity.com/paper/2503.17657/) (2025) - [Similarity-Guided Diffusion for Long-Gap Music Inpainting](https://mlsanity.com/paper/2509.16342/) (2025) - [On the Error-Correcting Effects of Stochasticity in Discrete Diffusion](https://mlsanity.com/paper/2605.26582/) (2026) ## Abstract (arXiv) The diffusion model has shown remarkable success in computer vision, but it remains unclear whether the ODE-based probability flow or the SDE-based diffusion model is more superior and under what circumstances. Comparing the two is challenging due to dependencies on data distributions, score training, and other numerical issues. In this paper, we study the problem mathematically for two limiting scenarios: the zero diffusion (ODE) case and the large diffusion case. We first introduce a pulse-shape error to perturb the score function and analyze error accumulation of sampling quality, followed by a thorough analysis for generalization to arbitrary error. Our findings indicate that when the perturbation occurs at the end of the generative process, the ODE model outperforms the SDE model with a large diffusion coefficient. However, when the perturbation occurs earlier, the SDE model outperforms the ODE model, and we demonstrate that the error of sample generation due to such a pulse-shape perturbation is exponentially suppressed as the diffusion term's magnitude increases to infinity. Numerical validation of this phenomenon is provided using Gaussian, Gaussian mixture, and Swiss roll distribution, as well as realistic datasets like MNIST and CIFAR-10.