mlsanityData to 14 Oct 2026

Reducing Noise in GAN Training with Variance Reduced Extragradient

arXiv 1904.08598 ↗First posted 18 Apr 2019stat.ML cs.LG math.OCcited by 62

Not on the map. Why?

Abstract (arXiv)

We study the effect of the stochastic gradient noise on the training of generative adversarial networks (GANs) and show that it can prevent the convergence of standard game optimization methods, while the batch version converges. We address this issue with a novel stochastic variance-reduced extragradient (SVRE) optimization algorithm, which for a large class of games improves upon the previous convergence rates proposed in the literature. We observe empirically that SVRE performs similarly to a batch method on MNIST while being computationally cheaper, and that SVRE yields more stable GAN training on standard datasets.

Key facts. Cited by 62 papers in the map (data through 14 Oct 2026). First posted on arXiv 18 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 17; 2022: 20; 2023: 10; 2024: 5; 2025: 4; 2026: 6.

62 papers.

Sort:

By area:

  1. 2021Stochastic Variance Reduction for Variational Inequality Methodscited by 47
  2. 2021The Complexity of Nonconvex-Strongly-Concave Minimax Optimizationcited by 45
  3. 2021Stochastic Gradient Descent-Ascent and Consensus Optimization for Smooth Games: Convergence Analysis under Expected Co-coercivitycited by 43
  4. 2021Extragradient Method: O(1/K) Last-Iterate Convergence for Monotone Variational Inequalities and Connections With Cocoercivitycited by 39
  5. 2022Stochastic Gradient Descent-Ascent: Unified Theory and New Efficient Methodscited by 37
  6. 2021Stochastic Extragradient: General Analysis and Improved Ratescited by 31
  7. 2021Decentralized Local Stochastic Extra-Gradient for Variational Inequalitiescited by 25
  8. 2021Lower Complexity Bounds of Finite-Sum Optimization Problems: The Results and Constructioncited by 24
  9. 2021Fast Distributionally Robust Learning with Variance Reduced Min-Max Optimizationcited by 19
  10. 2022Stochastic Halpern Iteration with Variance Reduction for Stochastic Monotone Inclusionscited by 19
  11. 2022Tight Last-Iterate Convergence of the Extragradient and the Optimistic Gradient Descent-Ascent Algorithm for Constrained Monotone Variational Inequalitiescited by 19
  12. 2021On the Convergence of Stochastic Extragradient for Bilinear Games using Restarted Iteration Averagingcited by 18
38 more
  1. 2022Regularized Gradient Descent Ascent for Two-Player Zero-Sum Markov Gamescited by 18
  2. 2022No-Regret Learning in Games with Noisy Feedback: Faster Rates and Adaptivity via Learning Rate Separationcited by 17
  3. 2021Near Optimal Stochastic Algorithms for Finite-Sum Unbalanced Convex-Concave Minimax Optimizationcited by 16
  4. 2021Distributed Methods with Compressed Communication for Solving Variational Inequalities, with Theoretical Guaranteescited by 15
  5. 2022Sharper Rates for Separable Minimax and Finite Sum Optimization via Primal-Dual Extragradient Methodscited by 15
  6. 2022Optimal Algorithms for Decentralized Stochastic Variational Inequalitiescited by 14
  7. 2023SAN: Inducing Metrizability of GAN with Discriminative Normalized Linear Layercited by 14
  8. 2023First Order Methods with Markovian Noise: from Acceleration to Variational Inequalitiescited by 14
  9. 2021Unified Convergence Analysis for Adaptive Optimization with Moving Average Estimatorcited by 13
  10. 2024Stochastic Halpern iteration in normed spaces and applications to reinforcement learningcited by 12
  11. 2021Complex Momentum for Optimization in Gamescited by 11
  12. 2023Variance Reduced Halpern Iteration for Finite-Sum Monotone Inclusionscited by 11
  13. 2022Smooth Monotone Stochastic Variational Inequalities and Saddle Point Problems: A Surveycited by 10
  14. 2021Last-Iterate Convergence of Saddle-Point Optimizers via High-Resolution Differential Equationscited by 8
  15. 2022Existence and Estimation of Critical Batch Size for Training Generative Adversarial Networks with Two Time-Scale Update Rulecited by 8
  16. 2022Compression and Data Similarity: Combination of Two Techniques for Communication-Efficient Solving of Distributed Variational Inequalitiescited by 8
  17. 2023Similarity, Compression and Local Steps: Three Pillars of Efficient Communications for Distributed Variational Inequalitiescited by 7
  18. 2023Balancing Act: Constraining Disparate Impact in Sparse Modelscited by 7
  19. 2025Worst-case generation via minimax optimization in Wasserstein spacecited by 7
  20. 2023Stochastic Methods in Variational Inequalities: Ergodicity, Bias and Refinementscited by 6
  21. 2021On the Convergence Rate of Off-Policy Policy Optimization Methods with Density-Ratio Correctioncited by 5
  22. 2023Faster Stochastic Algorithms for Minimax Optimization under Polyak--Lojasiewicz Conditionscited by 5
  23. 2025Multiplayer Federated Learning: Reaching Equilibrium with Less Communicationcited by 5
  24. 2023The equivalence of dynamic and strategic stability under regularized learning in gamescited by 4
  25. 2021Generative Minimization Networks: Training GANs Without Competitioncited by 3
  26. 2022Decentralized Stochastic Variance Reduced Extragradient Methodcited by 3
  27. 2022On Scaled Methods for Saddle Point Problemscited by 3
  28. 2022Solving Constrained Variational Inequalities via a First-order Interior Point-based Methodcited by 3
  29. 2024Variance-Reduced Fast Krasnoselkii-Mann Methods for Finite-Sum Root-Finding Problemscited by 3
  30. 2024Multiple Greedy Quasi-Newton Methods for Saddle Point Problemscited by 3
  31. 2026Unbiased and Biased Variance-Reduced Forward-Reflected-Backward Splitting Methods for Stochastic Composite Inclusionscited by 3
  32. 2021CDMA: A Practical Cross-Device Federated Learning Algorithm for General Minimax Problemscited by 2
  33. 2022Recursive Reasoning in Minimax Games: A Level k Gradient Play Methodcited by 2
  34. 2022Nesterov Meets Optimism: Rate-Optimal Separable Minimax Optimizationcited by 2
  35. 2024Diffusion Stochastic Optimization for Min-Max Problemscited by 2
  36. 2022A Variance-Reduced Stochastic Accelerated Primal Dual Algorithmcited by 1
  37. 2022Semi-Implicit Hybrid Gradient Methods with Application to Adversarial Robustnesscited by 1
  38. 2022Continuous-time Analysis for Variational Inequalities: An Overview and Desideratacited by 1

50 most cited of 62 All 62 (JSON)