mlsanityData to 7 Oct 2026

MonoFlow: Rethinking Divergence GANs via the Perspective of Wasserstein Gradient Flows

arXiv 2302.01075 ↗First posted 2 Feb 2023stat.ML cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

The conventional understanding of adversarial training in generative adversarial networks (GANs) is that the discriminator is trained to estimate a divergence, and the generator learns to minimize this divergence. We argue that despite the fact that many variants of GANs were developed following this paradigm, the current theoretical understanding of GANs and their practical algorithms are inconsistent. In this paper, we leverage Wasserstein gradient flows which characterize the evolution of particles in the sample space, to gain theoretical insights and algorithmic inspiration of GANs. We introduce a unified generative modeling framework - MonoFlow: the particle evolution is rescaled via a monotonically increasing mapping of the log density ratio. Under our framework, adversarial training can be viewed as a procedure first obtaining MonoFlow's vector field via training the discriminator and the generator learns to draw the particle flow defined by the corresponding vector field. We also reveal the fundamental difference between variational divergence minimization and adversarial training. This analysis helps us to identify what types of generator loss functions can lead to the successful training of GANs and suggest that GANs may have more loss designs beyond the literature (e.g., non-saturated loss), as long as they realize MonoFlow. Consistent empirical studies are included to validate the effectiveness of our framework.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 33 of its 50 references that are in the map. First posted on arXiv 2 Feb 2023.

Builds on

33 of 50 references.

References · 33

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  4. 2022Generative Adversarial Networkscited by 5,603
  5. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  6. 2013Auto-Encoding Variational Bayescited by 3,980
  7. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  8. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  9. 2017Improved Training of Wasserstein GANscited by 2,223
  10. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  11. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,898
  12. 2016Least Squares Generative Adversarial Networkscited by 1,000
21 more
  1. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 575
  2. 2016f-GAN: Training Generative Neural Samplers using Variational Divergence Minimizationcited by 504
  3. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 498
  4. 2022Understanding Diffusion Models: A Unified Perspectivecited by 337
  5. 2016NIPS 2016 Tutorial: Generative Adversarial Networkscited by 284
  6. 2019Monte Carlo Gradient Estimation in Machine Learningcited by 281
  7. 2021How to Train Your Energy-Based Modelscited by 271
  8. 2016Unrolled Generative Adversarial Networkscited by 208
  9. 2015Training generative neural networks via Maximum Mean Discrepancy optimizationcited by 149
  10. 2017Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inferencecited by 97
  11. 2020A Survey on Generative Adversarial Networks: Variants, Applications, and Trainingcited by 97
  12. 2020Generalized Energy Based Modelscited by 73
  13. 2020Refining Deep Generative Models via Discriminator Gradient Flowcited by 45
  14. 2019Deep Generative Learning via Variational Gradient Flowcited by 32
  15. 2017Sobolev GANcited by 29
  16. 2016Generative Adversarial Nets from a Density Ratio Estimation Perspectivecited by 24
  17. 2016Learning to Draw Samples: With Application to Amortized MLE for Generative Adversarial Learningcited by 21
  18. 2020On the Convergence of Gradient Descent in GANs: MMD GAN As a Gradient Flowcited by 21
  19. 2021A Neural Tangent Kernel Perspective of GANscited by 21
  20. 2019Training Neural Networks for Likelihood/Density Ratio Estimationcited by 16
  21. 2018Composite Functional Gradient Learning of Generative Adversarial Modelscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 5; 2025: 2; 2026: 1.

12 papers.

  1. 2023One-step Diffusion with Distribution Matching Distillationcited by 978
  2. 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 648
  3. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
  4. 2023Unifying GANs and Score-Based Diffusion as Generative Particle Modelscited by 21
  5. 2026Gradient Flow Drifting: Generative Modeling via Wasserstein Gradient Flows of KDE-Approximated Divergencescited by 15
  6. 2023Minimizing f-Divergences by Interpolating Velocity Fieldscited by 8
  7. 2023Bridging the Gap Between Variational Inference and Wasserstein Gradient Flowscited by 8
  8. 2024Inclusive KL Gradient Flows: Otto-Wasserstein, Fisher-Rao-Gaussian, and Local-Estimator Dynamicscited by 4
  9. 2025Training Latent Diffusion Models with Interacting Particle Algorithmscited by 4
  10. 2024Forward-Euler time-discretization for Wasserstein gradient flows can be wrongcited by 3
  11. 2025Towards Understanding Gradient Dynamics of the Sliced-Wasserstein Distance via Critical Point Analysiscited by 3
  12. 2024Distribution Backtracking Builds A Faster Convergence Trajectory for Diffusion Distillationcited by 1