Continuous-Time Distributed Dynamic Programming for Networked Multi-Agent Markov Decision Processes
Not on the map. Why?
Abstract (arXiv)
The main goal of this paper is to investigate continuous-time distributed dynamic programming (DP) algorithms for networked multi-agent Markov decision problems (MAMDPs). In our study, we adopt a distributed multi-agent framework where individual agents have access only to their own rewards, lacking insights into the rewards of other agents. Moreover, each agent has the ability to share its parameters with neighboring agents through a communication network, represented by a graph. We first introduce a novel distributed DP, inspired by the distributed optimization method of Wang and Elia. Next, a new distributed DP is introduced through a decoupling process. The convergence of the DP algorithms is proved through systems and control perspectives. The study in this paper sets the stage for new distributed temporal different learning algorithms.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 9 of its 31 references that are in the map. First posted on arXiv 31 Jul 2023.
Builds on
9 of 31 references.
References · 9
- 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 555
- 2014EXTRA: An Exact First-Order Algorithm for Decentralized Consensus Optimizationcited by 160
- 2018Multi-Agent Reinforcement Learning via Double Averaging Primal-Dual Optimizationcited by 40
- 2019Optimization for Reinforcement Learning: From Single Agent to Cooperative Agentscited by 19
- 2013Distributed Policy Evaluation Under Multiple Behavior Strategiescited by 17
- 2021A Discrete-Time Switching System Analysis of Q-learningcited by 16
- 2018Multi-Agent Fully Decentralized Value Function Learning with Linear Convergence Ratescited by 13
- 2020Fully Asynchronous Policy Evaluation in Distributed Reinforcement Learning over Networkscited by 6
- 2019Fast Multi-Agent Temporal-Difference Learning via Homotopy Stochastic Primal-Dual Optimizationcited by 3
Built on by
Not cited yet.