mlsanityData to 7 Oct 2026

The Score-Difference Flow for Implicit Generative Modeling

arXiv 2304.12906 ↗First posted 25 Apr 2023cs.LG stat.MLcited by 9

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Implicit generative modeling (IGM) aims to produce samples of synthetic data matching the characteristics of a target data distribution. Recent work (e.g. score-matching networks, diffusion models) has approached the IGM problem from the perspective of pushing synthetic source data toward the target distribution via dynamical perturbations or flows in the ambient space. In this direction, we present the score difference (SD) between arbitrary target and source distributions as a flow that optimally reduces the Kullback-Leibler divergence between them. We apply the SD flow to convenient proxy distributions, which are aligned if and only if the original distributions are aligned. We demonstrate the formal equivalence of this formulation to denoising diffusion models under certain conditions. We also show that the training of generative adversarial networks includes a hidden data-optimization sub-problem, which induces the SD flow under certain choices of loss function when the discriminator is optimal. As a result, the SD flow provides a theoretical link between model classes that individually address the three challenges of the "generative modeling trilemma" -- high sample quality, mode coverage, and fast sampling -- thereby setting the stage for a unified approach.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 31 of its 60 references that are in the map. First posted on arXiv 25 Apr 2023.

Builds on

31 of 60 references.

Closest ancestors · 8

Same direction, newest first

  1. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  2. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANs15 Dec
  3. 2021Variational Diffusion Models1 Jul
  4. 2021Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modeling1 Jun
  5. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  7. 2020Denoising Diffusion Probabilistic Models19 Jun
  8. 2020Improved Techniques for Training Score-Based Generative Models16 Jun

Other references · 23

  1. 2022Generative Adversarial Networkscited by 5,603
  2. 2018Neural Ordinary Differential Equationscited by 3,733
  3. 2015Variational Inference with Normalizing Flowscited by 2,161
  4. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,315
  5. 2018FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Modelscited by 584
  6. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 498
  7. 2020Hopfield Networks is All You Needcited by 461
  8. 2016NIPS 2016 Tutorial: Generative Adversarial Networkscited by 284
  9. 2021How to Train Your Energy-Based Modelscited by 271
  10. 2016A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluationcited by 236
  11. 2021Likelihood Training of Schr"odinger Bridge using Forward-Backward SDEs Theorycited by 216
  12. 2019Maximum Mean Discrepancy Gradient Flowcited by 123
11 more
  1. 2015Measuring Sample Quality with Stein's Methodcited by 94
  2. 2020Interacting particle solutions of Fokker-Planck equations through gradient-log-density estimationcited by 87
  3. 2013Density Estimation in Infinite Dimensional Exponential Familiescited by 41
  4. 2019Deep Generative Learning via Variational Gradient Flowcited by 32
  5. 2021Projected Wasserstein gradient descent for high-dimensional Bayesian inferencecited by 24
  6. 2018Spread Divergencecited by 23
  7. 2014Optimal steering of a linear stochastic system to a final probability distributioncited by 22
  8. 2019Stein's Lemma for the Reparameterization Trick with Exponential Family Mixturescited by 18
  9. 2021Deterministic particle flows for constraining stochastic nonlinear systemscited by 8
  10. 2021Relative Entropy Gradient Sampler for Unnormalized Distributionscited by 5
  11. 2021Exploiting the Hidden Tasks of GANs: Making Implicit Subproblems Explicitcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 1; 2025: 0; 2026: 7.

9 papers.

  1. 2023One-step Diffusion with Distribution Matching Distillationcited by 978
  2. 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 648
  3. 2026Generative Drifting is Secretly Score Matching: a Spectral and Variational Perspectivecited by 17
  4. 2026A Unified View of Score-Based and Drifting Modelscited by 8
  5. 2026On the Wasserstein Gradient Flow Interpretation of Drifting Modelscited by 6
  6. 2026Kernel-Gradient Drifting Modelscited by 6
  7. 2026Finite-Particle Convergence Rates for Conservative and Non-Conservative Drifting Modelscited by 2
  8. 2026Identifiability and Stability of Generative Drifting in the Companion-Elliptic Kernel Familycited by 0
  9. 2026DriftSE: Speech Enhancement with Generative Driftingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023DiffFlow: A Unified SDE Framework for Score-Based Diffusion Models and Generative Adversarial Networks
  2. 2026A Unified Measure-Theoretic View of Diffusion, Score-Based, and Flow Matching Generative Models
  3. 2026Where the Score Lives: A Wavelet View of Diffusion
  4. 2021A Variational Perspective on Diffusion-Based Generative Models and Score Matching
  5. 2021Gotta Go Fast When Generating Data with Score-Based Models
  6. 2024Discrete vs. Continuous Trade-offs for Generative Models
  7. 2025Low-dimensional adaptation of diffusion models: Convergence in total variation
  8. 2025Adaptivity and Convergence of Probability Flow ODEs in Diffusion Generative Models