# Interpreting diffusion score matching using normalizing flow

arXiv: https://arxiv.org/abs/2107.10072 · HTML: https://mlsanity.com/paper/2107.10072/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 17 references that are in the map.
- First posted on arXiv 21 Jul 2021.

## Builds on (most cited first, 14 of 14)

- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models](https://mlsanity.com/paper/1810.01367/) (2018)
- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [Sliced Score Matching: A Scalable Approach to Density and Score Estimation](https://mlsanity.com/paper/1905.07088/) (2019)
- [A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluation](https://mlsanity.com/paper/1602.03253/) (2016)
- [A Kernel Test of Goodness of Fit](https://mlsanity.com/paper/1602.02964/) (2016)
- [Measuring Sample Quality with Stein's Method](https://mlsanity.com/paper/1506.03039/) (2015)
- [Normalizing Flows on Riemannian Manifolds](https://mlsanity.com/paper/1611.02304/) (2016)
- [Minimum Stein Discrepancy Estimators](https://mlsanity.com/paper/1906.08283/) (2019)
- [Learning deep kernels for exponential family densities](https://mlsanity.com/paper/1811.08357/) (2018)
- [Measuring Sample Quality with Diffusions](https://mlsanity.com/paper/1611.06972/) (2016)
- [Measure Transport with Kernel Stein Discrepancy](https://mlsanity.com/paper/2010.11779/) (2020)

## Built on by (most cited first, 9 of 9)

- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [Likelihood Training of Schr"odinger Bridge using Forward-Backward SDEs Theory](https://mlsanity.com/paper/2110.11291/) (2021)
- [Mathematical analysis of singularities in the diffusion model under the submanifold assumption](https://mlsanity.com/paper/2301.07882/) (2023)
- [Towards Healing the Blindness of Score Matching](https://mlsanity.com/paper/2209.07396/) (2022)
- [Entropy-based Training Methods for Scalable Neural Implicit Sampler](https://mlsanity.com/paper/2306.04952/) (2023)
- [Score-based generative models break the curse of dimensionality in learning a family of sub-Gaussian probability distributions](https://mlsanity.com/paper/2402.08082/) (2024)
- [Optimal Stochastic Trace Estimation in Generative Modeling](https://mlsanity.com/paper/2502.18808/) (2025)
- [ScoreCL: Augmentation-Adaptive Contrastive Learning via Score-Matching Function](https://mlsanity.com/paper/2306.04175/) (2023)
- [Insights into Closed-form IPM-GAN Discriminator Guidance for Diffusion Modeling](https://mlsanity.com/paper/2306.01654/) (2023)

## Abstract (arXiv)

Scoring matching (SM), and its related counterpart, Stein discrepancy (SD) have achieved great success in model training and evaluations. However, recent research shows their limitations when dealing with certain types of distributions. One possible fix is incorporating the original score matching (or Stein discrepancy) with a diffusion matrix, which is called diffusion score matching (DSM) (or diffusion Stein discrepancy (DSD)). However, the lack of interpretation of the diffusion limits its usage within simple distributions and manually chosen matrix. In this work, we plan to fill this gap by interpreting the diffusion matrix using normalizing flows. Specifically, we theoretically prove that DSM (or DSD) is equivalent to the original score matching (or Stein discrepancy) evaluated in the transformed space defined by the normalizing flow, where the diffusion matrix is the inverse of the flow's Jacobian matrix. In addition, we also build its connection to Riemannian manifolds and further extend it to continuous flows, where the change of DSM is characterized by an ODE.