mlsanityData to 7 Oct 2026

A Variational Inequality Perspective on Generative Adversarial Networks

arXiv 1802.10551 ↗First posted 28 Feb 2018cs.LG math.OC stat.MLcited by 144

Not on the map. Why?

Abstract (arXiv)

Generative adversarial networks (GANs) form a generative modeling approach known for producing appealing samples, but they are notably difficult to train. One common way to tackle this issue has been to propose new formulations of the GAN objective. Yet, surprisingly few studies have looked at optimization methods designed for this adversarial training. In this work, we cast GAN optimization problems in the general variational inequality framework. Tapping into the mathematical programming literature, we counter some common misconceptions about the difficulties of saddle point optimization and propose to extend techniques designed for variational inequalities to the training of GANs. We apply averaging, extrapolation and a computationally cheaper variant that we call extrapolation from the past to the stochastic gradient method (SGD) and Adam.

Key facts. Cited by 144 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 38; 2022: 38; 2023: 27; 2024: 16; 2025: 10; 2026: 15.

144 papers.

Sort:

By area:

  1. 2021An Introduction to Deep Generative Modelingcited by 104
  2. 2021Independent Policy Gradient Methods for Competitive Reinforcement Learningcited by 97
  3. 2023A Fully First-Order Method for Stochastic Bilevel Optimizationcited by 97
  4. 2022FedNest: Federated Bilevel, Minimax, and Compositional Optimizationcited by 60
  5. 2021Adaptive Learning in Continuous Games: Optimal Regret Bounds and Convergence to Nash Equilibriumcited by 51
  6. 2023On Penalty Methods for Nonconvex Bilevel Optimization and First-Order Stochastic Approximationcited by 50
  7. 2021Stochastic Variance Reduction for Variational Inequality Methodscited by 46
  8. 2021The Complexity of Nonconvex-Strongly-Concave Minimax Optimizationcited by 44
  9. 2021Stochastic Gradient Descent-Ascent and Consensus Optimization for Smooth Games: Convergence Analysis under Expected Co-coercivitycited by 43
  10. 2021Local Stochastic Gradient Descent Ascent: Convergence Analysis and Communication Efficiencycited by 40
  11. 2022Stochastic Gradient Descent-Ascent: Unified Theory and New Efficient Methodscited by 37
  12. 2021Complexity Lower Bounds for Nonconvex-Strongly-Concave Min-Max Optimizationcited by 34
38 more
  1. 2024A Sparsity Principle for Partially Observable Causal Representation Learningcited by 34
  2. 2023Free Lunch for Domain Adversarial Training: Environment Label Smoothingcited by 33
  3. 2021Near-optimal Local Convergence of Alternating Gradient Descent-Ascent for Minimax Optimizationcited by 31
  4. 2021Stochastic Extragradient: General Analysis and Improved Ratescited by 31
  5. 2022Controlled Sparsity via Constrained Optimization or: How I Learned to Stop Tuning Penalties and Love Constraintscited by 25
  6. 2022A unified stochastic approximation framework for learning in gamescited by 22
  7. 2021Tighter Analysis of Alternating Stochastic Gradient Method for Stochastic Nested Problemscited by 21
  8. 2023Escaping limit cycles: Global convergence for constrained nonconvex-nonconcave minimax problemscited by 21
  9. 2021Exploration-Exploitation in Multi-Agent Competition: Convergence with Bounded Rationalitycited by 20
  10. 2021No-Regret Dynamics in the Fenchel Game: A Unified Framework for Algorithmic Convex Optimizationcited by 20
  11. 2022The First Optimal Algorithm for Smooth and Strongly-Convex-Strongly-Concave Minimax Optimizationcited by 20
  12. 2023ReLOAD: Reinforcement Learning with Optimistic Ascent-Descent for Last-Iterate Convergence in Constrained MDPscited by 19
  13. 2021On the Convergence of Stochastic Extragradient for Bilinear Games using Restarted Iteration Averagingcited by 18
  14. 2022Tight Last-Iterate Convergence of the Extragradient and the Optimistic Gradient Descent-Ascent Algorithm for Constrained Monotone Variational Inequalitiescited by 18
  15. 2021Accelerated Primal-Dual Gradient Method for Smooth and Convex-Concave Saddle-Point Problems with Bilinear Couplingcited by 16
  16. 2022The ICML 2022 Expressive Vocalizations Workshop and Competition: Recognizing, Generating, and Personalizing Vocal Burstscited by 16
  17. 2022Nest Your Adaptive Algorithm for Parameter-Agnostic Nonconvex Minimax Optimizationcited by 16
  18. 2021Near Optimal Stochastic Algorithms for Finite-Sum Unbalanced Convex-Concave Minimax Optimizationcited by 15
  19. 2021Distributed Methods with Compressed Communication for Solving Variational Inequalities, with Theoretical Guaranteescited by 15
  20. 2022Generalized Optimistic Methods for Convex-Concave Saddle Point Problemscited by 15
  21. 2024Dealing with unbounded gradients in stochastic saddle-point optimizationcited by 15
  22. 2022Optimal Algorithms for Decentralized Stochastic Variational Inequalitiescited by 14
  23. 2022An Exponentially Converging Particle Method for the Mixed Nash Equilibrium of Continuous Gamescited by 14
  24. 2023Single-Call Stochastic Extragradient Methods for Structured Non-monotone Variational Inequalities: Improved Analysis under Weaker Conditionscited by 14
  25. 2023First Order Methods with Markovian Noise: from Acceleration to Variational Inequalitiescited by 14
  26. 2022Lifted Primal-Dual Method for Bilinearly Coupled Smooth Minimax Optimizationcited by 13
  27. 2022Clipped Stochastic Methods for Variational Inequalities with Heavy-Tailed Noisecited by 13
  28. 2021The Last-Iterate Convergence Rate of Optimistic Mirror Descent in Stochastic Variational Inequalitiescited by 12
  29. 2024On PI Controllers for Updating Lagrange Multipliers in Constrained Optimizationcited by 12
  30. 2021Influence Estimation for Generative Adversarial Networkscited by 11
  31. 2021Complex Momentum for Optimization in Gamescited by 11
  32. 2021Learning in Matrix Games can be Arbitrarily Complexcited by 11
  33. 2022Universal Gradient Descent Ascent Method for Nonconvex-Nonconcave Minimax Optimizationcited by 11
  34. 2024On the Complexity of First-Order Methods in Stochastic Bilevel Optimizationcited by 11
  35. 2022Smooth Monotone Stochastic Variational Inequalities and Saddle Point Problems: A Surveycited by 10
  36. 2022Zeroth-Order Alternating Gradient Descent Ascent Algorithms for a Class of Nonconvex-Nonconcave Minimax Problemscited by 10
  37. 2025Cooper: A Library for Constrained Optimization in Deep Learningcited by 10
  38. 2021On the One-sided Convergence of Adam-type Algorithms in Non-convex Non-concave Min-max Optimizationcited by 9

50 most cited of 144 All 144 (JSON)