# On Gradient-Based Learning in Continuous Games

arXiv: https://arxiv.org/abs/1804.05464 · HTML: https://mlsanity.com/paper/1804.05464/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Apr 2018.

## Built on by (most cited first, 30 of 48)

- [Evolving Curricula with Regret-Based Environment Design](https://mlsanity.com/paper/2203.01302/) (2022)
- [Patch-Mix Transformer for Unsupervised Domain Adaptation: A Game Perspective](https://mlsanity.com/paper/2303.13434/) (2023)
- [Who Leads and Who Follows in Strategic Classification?](https://mlsanity.com/paper/2106.12529/) (2021)
- [Faster Single-loop Algorithms for Minimax Optimization without Strong Concavity](https://mlsanity.com/paper/2112.05604/) (2021)
- [Gradient play in stochastic games: stationary points, convergence, and sample complexity](https://mlsanity.com/paper/2106.00198/) (2021)
- [The Confluence of Networks, Games and Learning](https://mlsanity.com/paper/2105.08158/) (2021)
- [Policy Gradient Methods Find the Nash Equilibrium in N-player General-sum Linear-quadratic Games](https://mlsanity.com/paper/2107.13090/) (2021)
- [Fast Distributionally Robust Learning with Variance Reduced Min-Max Optimization](https://mlsanity.com/paper/2104.13326/) (2021)
- [Neural Operators of Backstepping Controller and Observer Gain Functions for Reaction-Diffusion PDEs](https://mlsanity.com/paper/2303.10506/) (2023)
- [On the Convergence of No-Regret Learning Dynamics in Time-Varying Games](https://mlsanity.com/paper/2301.11241/) (2023)
- [Learning in Matrix Games can be Arbitrarily Complex](https://mlsanity.com/paper/2103.03405/) (2021)
- [Proper Laplacian Representation Learning](https://mlsanity.com/paper/2310.10833/) (2023)
- [First-Order Algorithms for Min-Max Optimization in Geodesic Metric Spaces](https://mlsanity.com/paper/2206.02041/) (2022)
- [Adversarial Environment Design via Regret-Guided Diffusion Models](https://mlsanity.com/paper/2410.19715/) (2024)
- [Minimax Optimization with Smooth Algorithmic Adversaries](https://mlsanity.com/paper/2106.01488/) (2021)
- [Last-Iterate Convergence of Saddle-Point Optimizers via High-Resolution Differential Equations](https://mlsanity.com/paper/2112.13826/) (2021)
- [A Game-Theoretic Approach to Multi-Agent Trust Region Optimization](https://mlsanity.com/paper/2106.06828/) (2021)
- [Scenario generation for market risk models using generative neural networks](https://mlsanity.com/paper/2109.10072/) (2021)
- [Domain Adversarial Training: A Game Perspective](https://mlsanity.com/paper/2202.05352/) (2022)
- [Multiplayer Federated Learning: Reaching Equilibrium with Less Communication](https://mlsanity.com/paper/2501.08263/) (2025)
- [Similarity-based cooperative equilibrium](https://mlsanity.com/paper/2211.14468/) (2022)
- [Learning to Control Unknown Strongly Monotone Games](https://mlsanity.com/paper/2407.00575/) (2024)
- [Understanding Model Selection For Learning In Strategic Environments](https://mlsanity.com/paper/2402.07588/) (2024)
- [Robust Forecasting for Robotic Control: A Game-Theoretic Approach](https://mlsanity.com/paper/2209.10802/) (2022)
- [PAPAL: A Provable PArticle-based Primal-Dual ALgorithm for Mixed Nash Equilibrium](https://mlsanity.com/paper/2303.00970/) (2023)
- [Last-Iterate Convergence of Adaptive Riemannian Gradient Descent for Equilibrium Computation](https://mlsanity.com/paper/2306.16617/) (2023)
- [Convergent Q-Learning for Infinite-Horizon General-Sum Markov Games through Behavioral Economics](https://mlsanity.com/paper/2508.08669/) (2025)
- [Second-Order Mirror Descent: Convergence in Games Beyond Averaging and Discounting](https://mlsanity.com/paper/2111.09982/) (2021)
- [Competitive Gradient Optimization](https://mlsanity.com/paper/2205.14232/) (2022)
- [Continuous-time Analysis for Variational Inequalities: An Overview and Desiderata](https://mlsanity.com/paper/2207.07105/) (2022)

## Abstract (arXiv)

We formulate a general framework for competitive gradient-based learning that encompasses a wide breadth of multi-agent learning algorithms, and analyze the limiting behavior of competitive gradient-based learning algorithms using dynamical systems theory. For both general-sum and potential games, we characterize a non-negligible subset of the local Nash equilibria that will be avoided if each agent employs a gradient-based learning algorithm. We also shed light on the issue of convergence to non-Nash strategies in general- and zero-sum games, which may have no relevance to the underlying game, and arise solely due to the choice of algorithm. The existence and frequency of such strategies may explain some of the difficulties encountered when using gradient descent in zero-sum games as, e.g., in the training of generative adversarial networks. To reinforce the theoretical contributions, we provide empirical results that highlight the frequency of linear quadratic dynamic games (a benchmark for multi-agent reinforcement learning) that admit global Nash equilibria that are almost surely avoided by policy gradient.