mlsanityData to 7 Oct 2026

A Gauss-Newton Approach for Min-Max Optimization in Generative Adversarial Networks

arXiv 2404.07172 ↗First posted 10 Apr 2024cs.LG cs.NA math.NAcited by 1

Not on the map. Why?

Abstract (arXiv)

A novel first-order method is proposed for training generative adversarial networks (GANs). It modifies the Gauss-Newton method to approximate the min-max Hessian and uses the Sherman-Morrison inversion formula to calculate the inverse. The method corresponds to a fixed-point method that ensures necessary contraction. To evaluate its effectiveness, numerical experiments are conducted on various datasets commonly used in image generation tasks, such as MNIST, Fashion MNIST, CIFAR10, FFHQ, and LSUN. Our method is capable of generating high-fidelity images with greater diversity across multiple datasets. It also achieves the highest inception score for CIFAR10 among all compared methods, including state-of-the-art second-order methods. Additionally, its execution time is comparable to that of first-order min-max methods.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 30 of its 42 references that are in the map. First posted on arXiv 10 Apr 2024.

Builds on

30 of 42 references.

References · 30

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  5. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  6. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  7. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  8. 2016Improved Techniques for Training GANscited by 2,938
  9. 2017Improved Training of Wasserstein GANscited by 2,223
  10. 2014Conditional Generative Adversarial Netscited by 2,152
  11. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  12. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
18 more
  1. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  2. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  3. 2023Scaling up GANs for Text-to-Image Synthesiscited by 492
  4. 2014New insights and perspectives on the natural gradient methodcited by 435
  5. 2019Implicit Regularization in Deep Matrix Factorizationcited by 402
  6. 2017Implicit Regularization in Matrix Factorizationcited by 318
  7. 2017Training GANs with Optimismcited by 217
  8. 2013Revisiting Natural Gradient for Deep Networkscited by 157
  9. 2018The Mechanics of n-Player Differentiable Gamescited by 118
  10. 2017The Numerics of GANscited by 104
  11. 2018The Limit Points of (Optimistic) Gradient Descent in Min-Max Optimizationcited by 98
  12. 2017Implicit Regularization in Deep Learningcited by 90
  13. 2017Implicit Regularization in Nonconvex Statistical Estimation: Gradient Descent Converges Linearly for Phase Retrieval, Matrix Completion, and Blind Deconvolutioncited by 63
  14. 2019Competitive Gradient Descentcited by 32
  15. 2019Implicit competitive regularization in GANscited by 14
  16. 2022Riemannian Hamiltonian methods for min-max optimization on manifoldscited by 12
  17. 2023Adaptive Consensus Optimization Method for GANscited by 3
  18. 2023Angle based dynamic learning rate for gradient descentcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026SignMuon: Communication-Efficient Distributed Muon Optimizationcited by 2