Scalable and Sample Efficient Distributed Policy Gradient Algorithms in Multi-Agent Networked Systems
Not on the map. Why?
Abstract (arXiv)
This paper studies a class of multi-agent reinforcement learning (MARL) problems where the reward that an agent receives depends on the states of other agents, but the next state only depends on the agent's own current state and action. We name it REC-MARL standing for REward-Coupled Multi-Agent Reinforcement Learning. REC-MARL has a range of important applications such as real-time access control and distributed power control in wireless networks. This paper presents a distributed policy gradient algorithm for REC-MARL. The proposed algorithm is distributed in two aspects: (i) the learned policy is a distributed policy that maps a local state of an agent to its local action and (ii) the learning/training is distributed, during which each agent updates its policy based on its own and neighbors' information. The learned algorithm achieves a stationary policy and its iterative complexity bounds depend on the dimension of local states and actions. The experimental results of our algorithm for the real-time access control and power control in wireless networks show that our policy significantly outperforms the state-of-the-art algorithms and well-known benchmarks.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 19 of its 38 references that are in the map. First posted on arXiv 13 Dec 2022.
Builds on
19 of 38 references.
References · 19
- 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
- 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
- 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
- 2017Counterfactual Multi-Agent Policy Gradientscited by 733
- 2020On the Global Convergence Rates of Softmax Policy Gradient Methodscited by 258
- 2018Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agentscited by 189
- 2019On the Theory of Policy Gradient Methods: Optimality, Approximation, and Distribution Shiftcited by 174
- 2019Finite-Time Error Bounds For Linear Stochastic Approximation and TD Learningcited by 165
- 2020Sample Efficient Reinforcement Learning with REINFORCEcited by 85
- 2020Multi-agent Reinforcement Learning for Networked System Controlcited by 51
- 2020Scalable Multi-Agent Reinforcement Learning for Networked Systems with Average Rewardcited by 46
- 2019On the Convergence of Model Free Learning in Mean Field Gamescited by 45
7 more
- 2020Multi-Agent Reinforcement Learning in Stochastic Networked Systemscited by 30
- 2021Sample and Communication-Efficient Decentralized Actor-Critic Algorithms with Finite-Time Analysiscited by 25
- 2019Scalable Reinforcement Learning for Multi-Agent Networked Systemscited by 22
- 2022Global Convergence of Localized Policy Iteration in Networked Multi-Agent Reinforcement Learningcited by 18
- 2017Online Learning in Weakly Coupled Markov Decision Processes: A Convergence Time Studycited by 14
- 2020Scalable Reinforcement Learning Policies for Multi-Agent Controlcited by 10
- 2021MAMRL: Exploiting Multi-agent Meta Reinforcement Learning in WAN Traffic Engineeringcited by 1
Built on by
Citing papers per year: 2023: 1; 2024: 1.
2 papers.