# FP-Diffusion: Improving Score-based Diffusion Models by Enforcing the Underlying Score Fokker-Planck Equation

arXiv: https://arxiv.org/abs/2210.04296 · HTML: https://mlsanity.com/paper/2210.04296/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 20 of its 51 references that are in the map.
- First posted on arXiv 9 Oct 2022.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 20 of 20)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [Denoising Diffusion Restoration Models](https://mlsanity.com/paper/2201.11793/) (2022)
- [Maximum Likelihood Training of Score-Based Diffusion Models](https://mlsanity.com/paper/2101.09258/) (2021)
- [Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modeling](https://mlsanity.com/paper/2106.01357/) (2021)
- [Sliced Score Matching: A Scalable Approach to Density and Score Estimation](https://mlsanity.com/paper/1905.07088/) (2019)
- [A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets](https://mlsanity.com/paper/1707.08819/) (2017)
- [Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design](https://mlsanity.com/paper/1902.00275/) (2019)
- [Score-Based Generative Models Detect Manifolds](https://mlsanity.com/paper/2206.01018/) (2022)
- [Maximum Likelihood Training for Score-Based Diffusion ODEs by High-Order Denoising Score Matching](https://mlsanity.com/paper/2206.08265/) (2022)
- [Estimating High Order Gradients of the Data Distribution by Denoising](https://mlsanity.com/paper/2111.04726/) (2021)
- [Self-Consistency of the Fokker-Planck Equation](https://mlsanity.com/paper/2206.00860/) (2022)
- [A Modern Analysis of Hutchinson's Trace Estimator](https://mlsanity.com/paper/2012.12895/) (2020)

## Built on by (most cited first, 30 of 36)

- [Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion](https://mlsanity.com/paper/2310.02279/) (2023)
- [Consistent Diffusion Meets Tweedie: Training Exact Ambient Diffusion Models with Noisy Data](https://mlsanity.com/paper/2404.10177/) (2024)
- [Characteristic Guidance: Non-linear Correction for Diffusion Model at Large Guidance Scale](https://mlsanity.com/paper/2312.07586/) (2023)
- [Training Unbiased Diffusion Models From Biased Dataset](https://mlsanity.com/paper/2403.01189/) (2024)
- [Consistent Sampling and Simulation: Molecular Dynamics with Energy-Based Diffusion Models](https://mlsanity.com/paper/2506.17139/) (2025)
- [Diffusion Models as Cartoonists: The Curious Case of High Density Regions](https://mlsanity.com/paper/2411.01293/) (2024)
- [Reward-Directed Score-Based Diffusion Models via q-Learning](https://mlsanity.com/paper/2409.04832/) (2024)
- [Learning normalized image densities via dual score matching](https://mlsanity.com/paper/2506.05310/) (2025)
- [Exploring the Optimal Choice for Generative Processes in Diffusion Models: Ordinary vs Stochastic Differential Equations](https://mlsanity.com/paper/2306.02063/) (2023)
- [Wasserstein proximal operators describe score-based generative models and resolve memorization](https://mlsanity.com/paper/2402.06162/) (2024)
- [Understanding Diffusion Models by Feynman's Path Integral](https://mlsanity.com/paper/2403.11262/) (2024)
- [Closing the ODE-SDE gap in score-based diffusion models through the Fokker-Planck equation](https://mlsanity.com/paper/2311.15996/) (2023)
- [Diffusion Rejection Sampling](https://mlsanity.com/paper/2405.17880/) (2024)
- [Improving Flow Matching by Aligning Flow Divergence](https://mlsanity.com/paper/2602.00869/) (2026)
- [What's the score? Automated Denoising Score Matching for Nonlinear Diffusions](https://mlsanity.com/paper/2407.07998/) (2024)
- [Malliavin Calculus for Score-based Diffusion Models](https://mlsanity.com/paper/2503.16917/) (2025)
- [Improving Vector-Quantized Image Modeling with Latent Consistency-Matching Diffusion](https://mlsanity.com/paper/2410.14758/) (2024)
- [Bellman Diffusion: Generative Modeling as Learning a Linear Operator in the Distribution Space](https://mlsanity.com/paper/2410.01796/) (2024)
- [HERO: Human-Feedback Efficient Reinforcement Learning for Online Diffusion Model Finetuning](https://mlsanity.com/paper/2410.05116/) (2024)
- [RMFlow: Refined Mean Flow by a Noise-Injection Step for Multimodal Generation](https://mlsanity.com/paper/2602.00849/) (2026)
- [Neural Shr"odinger Bridge Matching for Pansharpening](https://mlsanity.com/paper/2404.11416/) (2024)
- [Minimal Impact ControlNet: Advancing Multi-ControlNet Integration](https://mlsanity.com/paper/2506.01672/) (2025)
- [Learning Energy-Based Models from Stochastic Interpolants using Spatiotemporal Differences](https://mlsanity.com/paper/2605.26850/) (2026)
- [Enhancing Semantic Communication with Deep Generative Models -- An ICASSP Special Session Overview](https://mlsanity.com/paper/2309.02478/) (2023)
- [Efficient Denoising using Score Embedding in Score-based Diffusion Models](https://mlsanity.com/paper/2404.06661/) (2024)
- [Score Shocks: The Burgers Equation Structure of Diffusion Generative Models](https://mlsanity.com/paper/2604.07404/) (2026)
- [Operator-Informed Score Matching for Markov Diffusion Models](https://mlsanity.com/paper/2406.09084/) (2024)
- [Local Curvature Smoothing with Stein's Identity for Efficient Score Matching](https://mlsanity.com/paper/2412.03962/) (2024)
- [Some Optimizers are More Equal: Understanding the Role of Optimizers in Group Fairness](https://mlsanity.com/paper/2504.14882/) (2025)
- [Variance-Reduced Diffusion Sampling via Target Score Identity](https://mlsanity.com/paper/2601.01594/) (2026)

## Abstract (arXiv)

Score-based generative models (SGMs) learn a family of noise-conditional score functions corresponding to the data density perturbed with increasingly large amounts of noise. These perturbed data densities are linked together by the Fokker-Planck equation (FPE), a partial differential equation (PDE) governing the spatial-temporal evolution of a density undergoing a diffusion process. In this work, we derive a corresponding equation called the score FPE that characterizes the noise-conditional scores of the perturbed data densities (i.e., their gradients). Surprisingly, despite the impressive empirical performance, we observe that scores learned through denoising score matching (DSM) fail to fulfill the underlying score FPE, which is an inherent self-consistency property of the ground truth score. We prove that satisfying the score FPE is desirable as it improves the likelihood and the degree of conservativity. Hence, we propose to regularize the DSM objective to enforce satisfaction of the score FPE, and we show the effectiveness of this approach across various datasets.