mlsanityData to 7 Oct 2026

Nash Convergence of Gradient Dynamics in Iterated General-Sum Games

arXiv 1301.3892 ↗First posted 16 Jan 2013cs.GTcited by 18

Not on the map. Why?

Abstract (arXiv)

Multi-agent games are becoming an increasing prevalent formalism for the study of electronic commerce and auctions. The speed at which transactions can take place and the growing complexity of electronic marketplaces makes the study of computationally simple agents an appealing direction. In this work, we analyze the behavior of agents that incrementally adapt their strategy through gradient ascent on expected payoff, in the simple setting of two-player, two-action, iterated general-sum games, and present a surprising result. We show that either the agents will converge to Nash equilibrium, or if the strategies themselves do not converge, then their average payoffs will nevertheless converge to the payoffs of a Nash equilibrium.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Jan 2013.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 8; 2022: 2; 2023: 4; 2024: 2; 2025: 2.

18 papers.

Sort:

  1. 2021Contrasting Centralized and Decentralized Critics in Multi-Agent Reinforcement Learningcited by 54
  2. 2021Discovering Diverse Multi-Agent Strategic Behavior via Reward Randomizationcited by 35
  3. 2022Modeling Content Creator Incentives on Algorithm-Curated Platformscited by 35
  4. 2023Multi-agent Reinforcement Learning: A Comprehensive Surveycited by 30
  5. 2021Modeling the Second Player in Distributionally Robust Optimizationcited by 29
  6. 2021Policy Gradient Methods Find the Nash Equilibrium in N-player General-sum Linear-quadratic Gamescited by 19
  7. 2022Distributionally Robust Models with Parametric Likelihood Ratioscited by 17
  8. 2021Discretization Drift in Two-Player Gamescited by 14
  9. 2021Contextual Games: Multi-Agent Learning with Side Informationcited by 11
  10. 2024On Centralized Critics in Multi-Agent Reinforcement Learningcited by 9
  11. 2021A Game-Theoretic Approach to Multi-Agent Trust Region Optimizationcited by 7
  12. 2023Combining Tree-Search, Generative Models, and Nash Bargaining Concepts in Game-Theoretic Reinforcement Learningcited by 6
6 more
  1. 2021Preference Communication in Multi-Objective Normal-Form Gamescited by 3
  2. 2023Off-Policy Evaluation for Action-Dependent Non-Stationary Environmentscited by 2
  3. 2024Paths to Equilibrium in Gamescited by 1
  4. 2023Chaos persists in large-scale multi-agent learning despite adaptive learning ratescited by 0
  5. 2025Grouped Satisficing Paths in Pure Strategy Games: a Topological Perspectivecited by 0
  6. 2025Learnable Mixed Nash Equilibria are Collectively Rationalcited by 0