mlsanityData to 7 Oct 2026

Negative Momentum for Improved Game Dynamics

arXiv 1807.04740 ↗First posted 12 Jul 2018cs.LG stat.MLcited by 59

Not on the map. Why?

Abstract (arXiv)

Games generalize the single-objective optimization paradigm by introducing different objective functions for different players. Differentiable games often proceed by simultaneous or alternating gradient updates. In machine learning, games are gaining new importance through formulations like generative adversarial networks (GANs) and actor-critic systems. However, compared to single-objective optimization, game dynamics are more complex and less understood. In this paper, we analyze gradient-based methods with momentum on simple games. We prove that alternating updates are more stable than simultaneous updates. Next, we show both theoretically and empirically that alternating gradient updates with a negative momentum term achieves convergence in a difficult toy adversarial problem, but also on the notoriously difficult to train saturating GANs.

Key facts. Cited by 59 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 Jul 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 19; 2022: 18; 2023: 4; 2024: 5; 2025: 9; 2026: 4.

59 papers.

Sort:

By area:

  1. 2021Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalizationcited by 220
  2. 2021Independent Policy Gradient Methods for Competitive Reinforcement Learningcited by 97
  3. 2023ATT3D: Amortized Text-to-3D Object Synthesiscited by 87
  4. 2021Near-optimal Local Convergence of Alternating Gradient Descent-Ascent for Minimax Optimizationcited by 31
  5. 2021Meta-Learning to Improve Pre-Trainingcited by 27
  6. 2024One-Shot Safety Alignment for Large Language Models via Optimal Dualizationcited by 24
  7. 2021Exploration-Exploitation in Multi-Agent Competition: Convergence with Bounded Rationalitycited by 20
  8. 2022Tight Last-Iterate Convergence of the Extragradient and the Optimistic Gradient Descent-Ascent Algorithm for Constrained Monotone Variational Inequalitiescited by 18
  9. 2021Accelerated Primal-Dual Gradient Method for Smooth and Convex-Concave Saddle-Point Problems with Bilinear Couplingcited by 16
  10. 2022Clipped Stochastic Methods for Variational Inequalities with Heavy-Tailed Noisecited by 13
  11. 2024On PI Controllers for Updating Lagrange Multipliers in Constrained Optimizationcited by 12
  12. 2021Complex Momentum for Optimization in Gamescited by 11
38 more
  1. 2022First-Order Algorithms for Min-Max Optimization in Geodesic Metric Spacescited by 10
  2. 2022Sampling without Replacement Leads to Faster Rates in Finite-Sum Minimax Optimizationcited by 10
  3. 2022Smooth Monotone Stochastic Variational Inequalities and Saddle Point Problems: A Surveycited by 10
  4. 2021On the One-sided Convergence of Adam-type Algorithms in Non-convex Non-concave Min-max Optimizationcited by 9
  5. 2022Learning (Very) Simple Generative Models Is Hardcited by 9
  6. 2024MomentumSMoE: Integrating Momentum into Sparse Mixture of Expertscited by 9
  7. 2025Negative Stepsizes Make Gradient-Descent-Ascent Convergecited by 9
  8. 2021Last-Iterate Convergence of Saddle-Point Optimizers via High-Resolution Differential Equationscited by 8
  9. 2021Lyapunov Exponents for Diversity in Differentiable Gamescited by 8
  10. 2021Direct-Search for a Class of Stochastic Min-Max Problemscited by 7
  11. 2022Domain Adversarial Training: A Game Perspectivecited by 7
  12. 2022Minimax Optimality (Probably) Doesn't Imply Distribution Learning for GANscited by 6
  13. 2025Score Distillation Sampling for Audio: Source Separation, Synthesis, and Beyondcited by 5
  14. 2021On the Initialization for Convex-Concave Min-max Problemscited by 4
  15. 2021Accelerated Proximal Alternating Gradient-Descent-Ascent for Nonconvex Minimax Machine Learningcited by 4
  16. 2022Nash, Conley, and Computation: Impossibility and Incompleteness in Game Dynamicscited by 4
  17. 2022Optimal Extragradient-Based Bilinearly-Coupled Saddle-Point Optimizationcited by 4
  18. 2022STay-ON-the-Ridge: Guaranteed Convergence to Local Minimax Equilibrium in Nonconvex-Nonconcave Gamescited by 4
  19. 2023Communication-Efficient Gradient Descent-Accent Methods for Distributed Variational Inequalities: Unified Analysis and Local Updatescited by 4
  20. 2021Generative Minimization Networks: Training GANs Without Competitioncited by 3
  21. 2022On Scaled Methods for Saddle Point Problemscited by 3
  22. 2023Optimal Guarantees for Algorithmic Reproducibility and Gradient Complexity in Convex Optimizationcited by 3
  23. 2025Continuous-Time Analysis of Heavy Ball Momentum in Min-Max Gamescited by 3
  24. 2021Relaxed Conditional Image Transfer for Semi-supervised Domain Adaptationcited by 2
  25. 2021Local AdaGrad-Type Algorithm for Stochastic Convex-Concave Optimizationcited by 2
  26. 2024Diffusion Stochastic Optimization for Min-Max Problemscited by 2
  27. 2025Optimism Without Regularization: Constant Regret in Zero-Sum Gamescited by 2
  28. 2025Expert Merging in Sparse Mixture of Experts with Nash Bargainingcited by 2
  29. 2021Consensus Multiplicative Weights Update: Learning to Learn using Projector-based Game Signaturescited by 1
  30. 2022Conjugate Gradient Method for Generative Adversarial Networkscited by 1
  31. 2022Competitive Gradient Optimizationcited by 1
  32. 2022Recursive Reasoning in Minimax Games: A Level k Gradient Play Methodcited by 1
  33. 2022When is Momentum Extragradient Optimal? A Polynomial-Based Analysiscited by 1
  34. 2024Statistical Mechanics of Min-Max Problemscited by 1
  35. 2021Constants of Motion: The Antidote to Chaos in Optimization and Game Dynamicscited by 0
  36. 2021GDA-AM: On the effectiveness of solving minimax optimization via Anderson Accelerationcited by 0
  37. 2022Three-Player Game Training Dynamicscited by 0
  38. 2022Iterative Minimax Games with Coupled Linear Constraintscited by 0

50 most cited of 59 All 59 (JSON)