mlsanityData to 14 Oct 2026

Convergence of flow-based generative models via proximal gradient descent in Wasserstein space

arXiv 2310.17582 ↗First posted 26 Oct 2023stat.ML cs.LG math.OCcited by 34

Not on the map. Why?

Abstract (arXiv)

Flow-based generative models enjoy certain advantages in computing the data generation and the likelihood, and have recently shown competitive empirical performance. Compared to the accumulating theoretical studies on related score-based diffusion models, analysis of flow-based models, which are deterministic in both forward (data-to-noise) and reverse (noise-to-data) directions, remain sparse. In this paper, we provide a theoretical guarantee of generating data distribution by a progressive flow model, the so-called JKO flow model, which implements the Jordan-Kinderleherer-Otto (JKO) scheme in a normalizing flow network. Leveraging the exponential convergence of the proximal gradient descent (GD) in Wasserstein space, we prove the Kullback-Leibler (KL) guarantee of data generation by a JKO flow model to be O(varepsilon2) when using N lesssim log (1/varepsilon) many JKO steps (N Residual Blocks in the flow) where varepsilon is the error in the per-step first-order condition. The assumption on data density is merely a finite second moment, and the theory extends to data distributions without density and when there are inversion errors in the reverse process where we obtain KL-W2 mixed error guarantees. The non-asymptotic convergence rate of the JKO-type W2-proximal GD is proved for a general class of convex objective functionals that includes the KL divergence as a special case, which can be of independent interest. The analysis framework can extend to other first-order Wasserstein optimization schemes applied to flow-based generative models.

Key facts. Cited by 34 papers in the map (data through 14 Oct 2026). Builds on 51 of its 79 references that are in the map. First posted on arXiv 26 Oct 2023.

Builds on

51 of 79 references.

References · 51

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  4. 2022Flow Matching for Generative Modelingcited by 5,702
  5. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  6. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  7. 2018Neural Ordinary Differential Equationscited by 3,779
  8. 2017Improved Training of Wasserstein GANscited by 2,235
  9. 2016Density estimation using Real NVPcited by 2,042
  10. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,545
  11. 2014NICE: Non-linear Independent Components Estimationcited by 1,058
  12. 2019An Introduction to Variational Autoencoderscited by 1,000
38 more
  1. 2022Building Normalizing Flows with Stochastic Interpolantscited by 995
  2. 2023Stochastic Interpolants: A Unifying Framework for Flows and Diffusionscited by 939
  3. 2021Maximum Likelihood Training of Score-Based Diffusion Modelscited by 672
  4. 2018FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Modelscited by 594
  5. 2016f-GAN: Training Generative Neural Samplers using Variational Divergence Minimizationcited by 506
  6. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptionscited by 445
  7. 2016Input Convex Neural Networkscited by 414
  8. 2018Invertible Residual Networkscited by 365
  9. 2022Rectified Flow: A Marginal Preserving Approach to Optimal Transportcited by 283
  10. 2022Convergence of denoising diffusion models under the manifold hypothesiscited by 242
  11. 2023Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localizationcited by 235
  12. 2020How to train your neural ODE: the world of Jacobian and kinetic regularizationcited by 228
  13. 2022Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptionscited by 219
  14. 2022Convergence of score-based generative modeling for general data distributionscited by 181
  15. 2022Convergence for score-based generative modeling with polynomial complexitycited by 177
  16. 2023The probability flow ODE is provably fastcited by 146
  17. 2020OT-Flow: Fast and Accurate Continuous Normalizing Flows via Optimal Transportcited by 143
  18. 2019Theoretical guarantees for sampling and inference in generative models with latent diffusionscited by 97
  19. 2019A Machine Learning Framework for Solving High-Dimensional Mean Field Game and Mean Field Control Problemscited by 97
  20. 2023Error Bounds for Flow Matching Methodscited by 91
  21. 2022Variational inference via Wasserstein gradient flowscited by 85
  22. 2023Towards Faster Non-Asymptotic Convergence for Diffusion-Based Generative Modelscited by 85
  23. 2021Large-Scale Wasserstein Gradient Flowscited by 77
  24. 2023Restoration-Degradation Beyond Linear Diffusions: A Non-Asymptotic Analysis For DDIM-Type Samplerscited by 74
  25. 2021Optimizing Functionals on the Space of Probabilities with Input Convex Neural Networkscited by 53
  26. 2022Normalizing flow neural networks by JKO schemecited by 39
  27. 2021An error analysis of generative adversarial networks for learning distributionscited by 36
  28. 2023Forward-backward Gaussian variational inference via JKO in the Bures-Wasserstein Spacecited by 36
  29. 2017On the ability of neural nets to express distributionscited by 26
  30. 2019Classification Logit Two-sample Testing by Neural Networkscited by 26
  31. 2021Neural Estimation of Statistical Divergencescited by 25
  32. 2023Improved Convergence of Score-Based Diffusion Models via Prediction-Correctioncited by 24
  33. 2023Distribution learning via neural differential equations: a nonparametric statistical perspectivecited by 22
  34. 2021On the capacity of deep generative networks for approximating distributionscited by 19
  35. 2022Bridging Mean-Field Games and Normalizing Flows with Trajectory Regularizationcited by 17
  36. 2022Taming Hyperparameter Tuning in Continuous Normalizing Flows Using the JKO Schemecited by 17
  37. 2023Computing high-dimensional optimal transport by flow neural networkscited by 8
  38. 2022Invertible Neural Networks for Graph Predictioncited by 6

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 14; 2025: 13; 2026: 6.

34 papers.

Sort:

By area:

  1. 2024A Sharp Convergence Theory for The Probability Flow ODEs of Diffusion Modelscited by 65
  2. 2024Convergence Analysis for General Probability Flow ODEs of Diffusion Models in Wasserstein Distancescited by 56
  3. 2025Low-dimensional adaptation of diffusion models: Convergence in total variationcited by 39
  4. 2024Learning stochastic dynamics from snapshots through regularized unbalanced optimal transportcited by 33
  5. 2024Provable Acceleration for Diffusion Models under Minimal Assumptionscited by 27
  6. 2024Convergence of Continuous Normalizing Flows for Learning Probability Distributionscited by 26
  7. 2024Convergence of Score-Based Discrete Diffusion Models: A Discrete-Time Analysiscited by 24
  8. 2024Scalable Wasserstein Gradient Flow for Generative Modeling through Unbalanced Optimal Transportcited by 23
  9. 2024Broadening Target Distributions for Accelerated Diffusion Models via a Novel Analysis Approachcited by 19
  10. 2025Solving Inverse Problems via Diffusion-Based Priors: An Approximation-Free Ensemble Sampling Approachcited by 18
  11. 2025Faster Diffusion Models via Higher-Order Approximationcited by 16
  12. 2023Flow-based Distributionally Robust Optimizationcited by 13
22 more
  1. 2024Annealing Flow Generative Models Towards Sampling High-Dimensional and Multi-Modal Distributionscited by 11
  2. 2025Integrating Dynamical Systems Modeling with Spatiotemporal scRNA-seq Data Analysiscited by 9
  3. 2025Learning of Population Dynamics: Inverse Optimization Meets JKO Schemecited by 9
  4. 2024Theory on Score-Mismatched Diffusion Models and Zero-Shot Conditional Samplerscited by 7
  5. 2025Worst-case generation via minimax optimization in Wasserstein spacecited by 7
  6. 2024Posterior sampling via Langevin dynamics based on generative priorscited by 5
  7. 2024Local Flow Matching Generative Modelscited by 5
  8. 2025Assured autonomy: How operations research powers and orchestrates generative AI systemscited by 5
  9. 2024Linear convergence of proximal descent schemes on the Wasserstein spacecited by 4
  10. 2025Pathway to O(√d) Complexity bound under Wasserstein metric of flow-based modelscited by 4
  11. 2026Generative models for decision-making under distributional shiftcited by 4
  12. 2025Dimension-free error estimate for diffusion model and optimal schedulingcited by 3
  13. 2025Convergence Analysis of the Wasserstein Proximal Algorithm beyond Geodesic Convexitycited by 2
  14. 2025Flow-based generative models as iterative algorithms in probability spacecited by 2
  15. 2026Learn to Evolve: Self-supervised Neural JKO Operator for Wasserstein Gradient Flowcited by 2
  16. 2024New algorithms for sampling and diffusion modelscited by 1
  17. 2026Diffusion Models for Adaptive Sequential Data Generationcited by 1
  18. 2025Embedding Empirical Distributions for Computing Optimal Transport Mapscited by 0
  19. 2025Liouville PDE-based sliced-Wasserstein flowcited by 0
  20. 2026Analyzing and Improving Diffusion Models for Time-Series Data Imputation: A Proximal Recursion Perspectivecited by 0
  21. 2026Weighted Wasserstein Barycenter of Gaussian Processes for exotic Bayesian Optimization taskscited by 0
  22. 2026A Unified Kullback--Leibler Divergence Analysis of Generative Diffusion Models via Entropy Production Ratecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Convergence Analysis of Probability Flow ODE for Score-based Generative Models
  2. 2025Non-asymptotic error bounds for probability flow ODEs under weak log-concavity
  3. 2025Adaptivity and Convergence of Probability Flow ODEs in Diffusion Generative Models
  4. 2026One-Step Generative Modeling via Wasserstein Gradient Flows
  5. 2025Wasserstein Convergence of Score-based Generative Models under Semiconvexity and Discontinuous Gradients
  6. 2026A Unified Measure-Theoretic View of Diffusion, Score-Based, and Flow Matching Generative Models
  7. 2022Score-based Generative Modeling Secretly Minimizes the Wasserstein Distance
  8. 2023On diffusion-based generative models and their error bounds: The log-concave case with full convergence estimates