# Decentralized Multi-Agent Reinforcement Learning with Networked Agents: Recent Advances

arXiv: https://arxiv.org/abs/1912.03821 · HTML: https://mlsanity.com/paper/1912.03821/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Dec 2019.

## Built on by (most cited first, 23 of 23)

- [A Review of Cooperation in Multi-agent Learning](https://mlsanity.com/paper/2312.05162/) (2023)
- [MALib: A Parallel Framework for Population-based Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2106.07551/) (2021)
- [Distributed Reinforcement Learning for Robot Teams: A Review](https://mlsanity.com/paper/2204.03516/) (2022)
- [Decentralized Cooperative Reinforcement Learning with Hierarchical Information Structure](https://mlsanity.com/paper/2111.00781/) (2021)
- [Common Information based Approximate State Representations in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2110.12603/) (2021)
- [Learning Decentralized Partially Observable Mean Field Control for Artificial Collective Behavior](https://mlsanity.com/paper/2307.06175/) (2023)
- [MDPGT: Momentum-based Decentralized Policy Gradient Tracking](https://mlsanity.com/paper/2112.02813/) (2021)
- [Model-based Reinforcement Learning for Service Mesh Fault Resiliency in a Web Application-level](https://mlsanity.com/paper/2110.13621/) (2021)
- [Multi-Agent Reinforcement Learning: Methods, Applications, Visionary Prospects, and Challenges](https://mlsanity.com/paper/2305.10091/) (2023)
- [Deception in Social Learning: A Multi-Agent Reinforcement Learning Perspective](https://mlsanity.com/paper/2106.05402/) (2021)
- [Interactive Medical Image Segmentation with Self-Adaptive Confidence Calibration](https://mlsanity.com/paper/2111.07716/) (2021)
- [Reward-Sharing Relational Networks in Multi-Agent Reinforcement Learning as a Framework for Emergent Behavior](https://mlsanity.com/paper/2207.05886/) (2022)
- [Coach-assisted Multi-Agent Reinforcement Learning Framework for Unexpected Crashed Agents](https://mlsanity.com/paper/2203.08454/) (2022)
- [Scalable regret for learning to control network-coupled subsystems with unknown dynamics](https://mlsanity.com/paper/2108.07970/) (2021)
- [Multi-agent Natural Actor-critic Reinforcement Learning Algorithms](https://mlsanity.com/paper/2109.01654/) (2021)
- [Decentralised Q-Learning for Multi-Agent Markov Decision Processes with a Satisfiability Criterion](https://mlsanity.com/paper/2311.12613/) (2023)
- [Scalable Interactive Machine Learning for Future Command and Control](https://mlsanity.com/paper/2402.06501/) (2024)
- [Distributed Algorithms for Linearly-Solvable Optimal Control in Networked Multi-Agent Systems](https://mlsanity.com/paper/2102.09104/) (2021)
- [Networked Communication for Decentralised Agents in Mean-Field Games](https://mlsanity.com/paper/2306.02766/) (2023)
- [Mini Honor of Kings: A Lightweight Environment for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2406.03978/) (2024)
- [Single-Loop Federated Actor-Critic across Heterogeneous Environments](https://mlsanity.com/paper/2412.14555/) (2024)
- [Core-Halo Decomposition: Decentralizing Large-Scale Fixed-Point Problems](https://mlsanity.com/paper/2605.08681/) (2026)
- [PC3D: Zero-Shot Cooperation Across Variable Rosters via Personalized Context Distillation](https://mlsanity.com/paper/2605.10377/) (2026)

## Abstract (arXiv)

Multi-agent reinforcement learning (MARL) has long been a significant and everlasting research topic in both machine learning and control. With the recent development of (single-agent) deep RL, there is a resurgence of interests in developing new MARL algorithms, especially those that are backed by theoretical analysis. In this paper, we review some recent advances a sub-area of this topic: decentralized MARL with networked agents. Specifically, multiple agents perform sequential decision-making in a common environment, without the coordination of any central controller. Instead, the agents are allowed to exchange information with their neighbors over a communication network. Such a setting finds broad applications in the control and operation of robots, unmanned vehicles, mobile sensor networks, and smart grid. This review is built upon several our research endeavors in this direction, together with some progresses made by other researchers along the line. We hope this review to inspire the devotion of more research efforts to this exciting yet challenging area.