Stability of Multi-Agent Learning: Convergence in Network Games with Many Players
Not on the map. Why?
Abstract (arXiv)
The behaviour of multi-agent learning in many player games has been shown to display complex dynamics outside of restrictive examples such as network zero-sum games. In addition, it has been shown that convergent behaviour is less likely to occur as the number of players increase. To make progress in resolving this problem, we study Q-Learning dynamics and determine a sufficient condition for the dynamics to converge to a unique equilibrium in any network game. We find that this condition depends on the nature of pairwise interactions and on the network structure, but is explicitly independent of the total number of agents in the game. We evaluate this result on a number of representative network games and show that, under suitable network conditions, stable learning dynamics can be achieved with an arbitrary number of agents.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 55 references that are in the map. First posted on arXiv 26 Jul 2023.
Builds on
16 of 55 references.
References · 16
- 2022Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learningcited by 131
- 2017Cycles in adversarial regularized learningcited by 124
- 2016Learning in games with continuous action sets and unknown payoff functionscited by 81
- 2022On Last-Iterate Convergence Beyond Zero-Sum Gamescited by 31
- 2021Principled Exploration via Optimistic Bootstrapping and Backward Inductioncited by 26
- 2019Last-iterate convergence rates for min-max optimizationcited by 25
- 2021Exploration-Exploitation in Multi-Agent Competition: Convergence with Bounded Rationalitycited by 21
- 2020Exploration-Exploitation in Multi-Agent Learning: Catastrophe Theory Meets Game Theorycited by 17
- 2019Learning Nash Equilibria in Monotone Gamescited by 11
- 2021Learning in Matrix Games can be Arbitrarily Complexcited by 11
- 2015Finite composite games: Equilibria and dynamicscited by 9
- 2017A variational inequality framework for network games: Existence, uniqueness, convergence and sensitivity analysiscited by 9
4 more
- 2023Asymptotic Convergence and Performance of Multi-Agent Q-Learning Dynamicscited by 8
- 2021Learning in nonatomic games, Part I: Finite action spaces and population gamescited by 7
- 2020Exponential Convergence of Gradient Methods in Concave Network Zero-sum Gamescited by 4
- 2021Poincar'e-Bendixson Limit Sets in Multi-Agent Learningcited by 3
Built on by
Not cited yet.