# Survey of Recent Multi-Agent Reinforcement Learning Algorithms Utilizing Centralized Training

arXiv: https://arxiv.org/abs/2107.14316 · HTML: https://mlsanity.com/paper/2107.14316/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 47 references that are in the map.
- First posted on arXiv 29 Jul 2021.

## Builds on (most cited first, 25 of 25)

- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Counterfactual Multi-Agent Policy Gradients](https://mlsanity.com/paper/1705.08926/) (2017)
- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1803.11485/) (2018)
- [Learning to Communicate with Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1605.06676/) (2016)
- [Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms](https://mlsanity.com/paper/1911.10635/) (2019)
- [Actor-Attention-Critic for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1810.02912/) (2018)
- [Mean Field Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1802.05438/) (2018)
- [Weighted QMIX: Expanding Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2006.10800/) (2020)
- [Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents](https://mlsanity.com/paper/1802.08757/) (2018)
- [MAVEN: Multi-Agent Variational Exploration](https://mlsanity.com/paper/1910.07483/) (2019)
- [Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observability](https://mlsanity.com/paper/1703.06182/) (2017)
- [Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1702.08887/) (2017)
- [Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/1906.04737/) (2019)
- [Parametrized Deep Q-Networks Learning: Reinforcement Learning with Discrete-Continuous Hybrid Action Space](https://mlsanity.com/paper/1810.06394/) (2018)
- [Learning Policy Representations in Multiagent Systems](https://mlsanity.com/paper/1806.06464/) (2018)
- [Deep Multi-Agent Reinforcement Learning with Discrete-Continuous Hybrid Action Spaces](https://mlsanity.com/paper/1903.04959/) (2019)
- [Forethought and Hindsight in Credit Assignment](https://mlsanity.com/paper/2010.13685/) (2020)
- [F2A2: Flexible Fully-decentralized Approximate Actor-critic for Cooperative Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2004.11145/) (2020)
- [Distributed Deep Q-Learning](https://mlsanity.com/paper/1508.04186/) (2015)
- [Robust Multi-agent Counterfactual Prediction](https://mlsanity.com/paper/1904.02235/) (2019)
- [Learning Nash Equilibria in Zero-Sum Stochastic Games via Entropy-Regularized Policy Approximation](https://mlsanity.com/paper/2009.00162/) (2020)
- [Image-Audio Encoding to Improve C2 Decision-Making in Multi-Domain Environment](https://mlsanity.com/paper/2106.00787/) (2021)
- [IoT Solutions with Multi-Sensor Fusion and Signal-Image Encoding for Secure Data Transfer and Decision Making](https://mlsanity.com/paper/2106.01497/) (2021)
- [Heterogeneous Noisy Short Signal Camouflage in Multi-Domain Environment Decision-Making](https://mlsanity.com/paper/2106.02044/) (2021)

## Built on by (most cited first, 16 of 16)

- [Causal Multi-Agent Reinforcement Learning: Review and Open Problems](https://mlsanity.com/paper/2111.06721/) (2021)
- [Transformers in Reinforcement Learning: A Survey](https://mlsanity.com/paper/2307.05979/) (2023)
- [LINDA: Multi-Agent Local Information Decomposition for Awareness of Teammates](https://mlsanity.com/paper/2109.12508/) (2021)
- [HCRide: Harmonizing Passenger Fairness and Driver Preference for Human-Centered Ride-Hailing](https://mlsanity.com/paper/2508.04811/) (2025)
- [Bilateral Deep Reinforcement Learning Approach for Better-than-human Car Following Model](https://mlsanity.com/paper/2203.04749/) (2022)
- [A Taxonomy of Architecture Options for Foundation Model-based Agents: Analysis and Decision Model](https://mlsanity.com/paper/2408.02920/) (2024)
- [Renaissance Robot: Optimal Transport Policy Fusion for Learning Diverse Skills](https://mlsanity.com/paper/2207.00978/) (2022)
- [Socially-Weighted Alignment: A Game-Theoretic Framework for Multi-Agent LLM Systems](https://mlsanity.com/paper/2602.14471/) (2026)
- [Strategic Maneuver and Disruption with Reinforcement Learning Approaches for Multi-Agent Coordination](https://mlsanity.com/paper/2203.09565/) (2022)
- [User-centric Heterogeneous-action Deep Reinforcement Learning for Virtual Reality in the Metaverse over Wireless Networks](https://mlsanity.com/paper/2302.01471/) (2023)
- [Orchestration of Emulator Assisted Mobile Edge Tuning for AI Foundation Models: A Multi-Agent Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2310.17492/) (2023)
- [Privacy-Engineered Value Decomposition Networks for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2311.06255/) (2023)
- [Emergent Behaviors in Multi-Agent Target Acquisition](https://mlsanity.com/paper/2212.07891/) (2022)
- [Virtual Reality in Metaverse over Wireless Networks with User-centered Deep Reinforcement Learning](https://mlsanity.com/paper/2303.04349/) (2023)
- [Cooperative Path Planning with Asynchronous Multiagent Reinforcement Learning](https://mlsanity.com/paper/2409.00754/) (2024)
- [GPEvac: GNN-Based PPO for Adaptive Evacuation Routing During Shooting Events](https://mlsanity.com/paper/2609.16163/) (2026)

## Abstract (arXiv)

Much work has been dedicated to the exploration of Multi-Agent Reinforcement Learning (MARL) paradigms implementing a centralized learning with decentralized execution (CLDE) approach to achieve human-like collaboration in cooperative tasks. Here, we discuss variations of centralized training and describe a recent survey of algorithmic approaches. The goal is to explore how different implementations of information sharing mechanism in centralized learning may give rise to distinct group coordinated behaviors in multi-agent systems performing cooperative tasks.