mlsanityData to 7 Oct 2026

WGAN with an Infinitely Wide Generator Has No Spurious Stationary Points

arXiv 2102.07541 ↗First posted 15 Feb 2021cs.LG math.OCcited by 1

Not on the map. Why?

Abstract (arXiv)

Generative adversarial networks (GAN) are a widely used class of deep generative models, but their minimax training dynamics are not understood very well. In this work, we show that GANs with a 2-layer infinite-width generator and a 2-layer finite-width discriminator trained with stochastic gradient ascent-descent have no spurious stationary points. We then show that when the width of the generator is finite but wide, there are no spurious stationary points within a ball whose radius becomes arbitrarily large (to cover the entire parameter space) as the width goes to infinity.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 32 of its 59 references that are in the map. First posted on arXiv 15 Feb 2021.

Builds on

32 of 59 references.

References · 32

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  2. 2017Improved Training of Wasserstein GANscited by 2,223
  3. 2017Visualizing the Loss Landscape of Neural Netscited by 1,126
  4. 2018Which Training Methods for GANs do actually Converge?cited by 672
  5. 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 655
  6. 2017Deep Neural Networks as Gaussian Processescited by 617
  7. 2018Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNscited by 585
  8. 2018A Mean Field View of the Landscape of Two-Layers Neural Networkscited by 463
  9. 2018On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transportcited by 422
  10. 2016NIPS 2016 Tutorial: Generative Adversarial Networkscited by 284
  11. 2017How to Escape Saddle Points Efficientlycited by 284
  12. 2018Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Datacited by 260
20 more
  1. 2015Escaping From Saddle Points --- Online Stochastic Gradient for Tensor Decompositioncited by 241
  2. 2019Mean-field theory of two-layers neural networks: dimension-free bounds and kernel limitcited by 187
  3. 2017No Spurious Local Minima in Nonconvex Low Rank Problems: A Unified Geometric Analysiscited by 127
  4. 2016Matrix Completion has No Spurious Local Minimumcited by 124
  5. 2018Mean Field Analysis of Neural Networks: A Central Limit Theoremcited by 107
  6. 2019Explaining Landscape Connectivity of Low-cost Solutions for Multilayer Netscited by 63
  7. 2020The Global Landscape of Neural Networks: An Overviewcited by 57
  8. 2018On the Convergence and Robustness of Training GANs with Regularized Optimal Transportcited by 51
  9. 2019Sparse Optimization on Measures with Over-parameterized Gradient Descentcited by 50
  10. 2018Finding Mixed Nash Equilibria of Generative Adversarial Networkscited by 48
  11. 2018On the loss landscape of a class of deep neural networks with no bad local valleyscited by 38
  12. 2019Landscape Connectivity and Dropout Stability of SGD Solutions for Over-parameterized Neural Networkscited by 35
  13. 2020A mean-field analysis of two-player zero-sum gamescited by 28
  14. 2018Understanding the Loss Surface of Neural Networks for Binary Classificationcited by 27
  15. 2018Adding One Neuron Can Eliminate All Bad Local Minimacited by 21
  16. 2020Towards a Better Global Loss Landscape of GANscited by 17
  17. 2018On the Benefit of Width for Neural Networks: Disappearance of Bad Basinscited by 16
  18. 2019SGD Learns One-Layer Networks in WGANscited by 16
  19. 2019Wasserstein GAN Can Perform PCAcited by 4
  20. 2019When Does Non-Orthogonal Tensor Decomposition Have No Spurious Local Minima?cited by 3

Built on by

Cited by year

Citing papers per year: 2022: 1.

1 paper.

  1. 2022Self-scalable Tanh (Stan): Faster Convergence and Better Generalization in Physics-informed Neural Networkscited by 5