A Scalable Game Theoretic Approach for Coordination of Multiple Dynamic Systems
On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization
Abstract (arXiv)
Learning in games provides a powerful framework to design control policies for self-interested agents that may be coupled through their dynamics, costs, or constraints. We consider the case where the dynamics of the coupled system can be modeled as a Markov potential game. In this case, distributed learning by the agents ensures that their control policies converge to a Nash equilibrium of this game. However, typical learning algorithms such as natural policy gradient require knowledge of the entire global state and actions of all the other agents, and may not be scalable as the number of agents grows. We show that by limiting the information flow to a local neighborhood of agents in the natural policy gradient algorithm, we can converge to a neighborhood of optimal policies. If the game can be designed through decomposing a global cost function of interest to a designer into local costs for the agents such that their policies at equilibrium optimize the global cost, this approach can be of interest to team coordination problems as well. We illustrate our approach through a sensor coverage problem.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 16 of its 28 references that are in the map. First posted on arXiv 17 Sep 2024.
Builds on
16 of 28 references.
References · 16
- 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 567
- 2021Independent Policy Gradient Methods for Competitive Reinforcement Learningcited by 101
- 2021Global Convergence of Multi-Agent Policy Gradient in Markov Potential Gamescited by 93
- 2021Decentralized Q-Learning in Zero-sum Markov Gamescited by 58
- 2022Independent Policy Gradient for Large-Scale Markov Potential Games: Sharper Rates, Function Approximation, and Game-Agnostic Convergencecited by 57
- 2020Scalable Multi-Agent Reinforcement Learning for Networked Systems with Average Rewardcited by 50
- 2021Independent Natural Policy Gradient Always Converges in Markov Potential Gamescited by 37
- 2021Learning in Nonzero-Sum Stochastic Games with Potentialscited by 26
- 2021Independent Learning in Stochastic Gamescited by 22
- 2022Independent and Decentralized Learning in Markov Potential Gamescited by 22
- 2020Graphon Mean Field Games and the GMFG Equationscited by 13
- 2022Fictitious Play in Markov Games with Single Controllercited by 10
4 more
- 2023Multi-Player Zero-Sum Markov Games with Networked Separable Interactionscited by 10
- 2020Dissipativity Tools for Convergence to Nash Equilibria in Population Gamescited by 2
- 2023Differential Equation Approximations for Population Games using Elementary Probabilitycited by 2
- 2022Epidemic Population Games And Evolutionary Dynamicscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Almost Sure Convergence of Networked Policy Gradient over Time-Varying Networks in Markov Potential Games
- 2025Policy Gradient with Self-Attention for Model-Free Distributed Nonlinear Multi-Agent Games ↗
- 2024Convergence of Decentralized Actor-Critic Algorithm in General-sum Markov Games
- 2025Learning Closed-Loop Parametric Nash Equilibria of Multi-Agent Collaborative Field Coverage ↗
- 2024Game Theory and Multi-Agent Reinforcement Learning : From Nash Equilibria to Evolutionary Dynamics
- 2024Independent RL for Cooperative-Competitive Agents: A Mean-Field Perspective
- 2026Internal State-Based Policy Gradient Methods for Partially Observable Markov Potential Games
- 2026Learning Approximate Nash Equilibria in Cooperative Multi-Agent Reinforcement Learning via Mean-Field Subsampling