# Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications

arXiv: https://arxiv.org/abs/1812.11794 · HTML: https://mlsanity.com/paper/1812.11794/

## Key facts

- Cited by 127 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Dec 2018.

## Built on by (most cited first, 30 of 127)

- [A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges](https://mlsanity.com/paper/2309.02473/) (2023)
- [Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospects](https://mlsanity.com/paper/2401.03428/) (2024)
- [LLM-based Multi-Agent Reinforcement Learning: Current and Future Directions](https://mlsanity.com/paper/2405.11106/) (2024)
- [Embodied AI Agents: Modeling the World](https://mlsanity.com/paper/2506.22355/) (2025)
- [Modeling Human Driving Behavior through Generative Adversarial Imitation Learning](https://mlsanity.com/paper/2006.06412/) (2020)
- [Deep Multiagent Reinforcement Learning: Challenges and Directions](https://mlsanity.com/paper/2106.15691/) (2021)
- [Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learning](https://mlsanity.com/paper/2107.04050/) (2021)
- [Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2408.09675/) (2024)
- [Deep Reinforcement Learning for Solving the Heterogeneous Capacitated Vehicle Routing Problem](https://mlsanity.com/paper/2110.02629/) (2021)
- [Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey](https://mlsanity.com/paper/2110.01411/) (2021)
- [Integrated Decision and Control: Towards Interpretable and Computationally Efficient Driving Intelligence](https://mlsanity.com/paper/2103.10290/) (2021)
- [A Survey on Context-Aware Multi-Agent Systems: Techniques, Challenges and Future Directions](https://mlsanity.com/paper/2402.01968/) (2024)
- [Cooperative Multi-Agent Deep Reinforcement Learning for Reliable Surveillance via Autonomous Multi-UAV Control](https://mlsanity.com/paper/2201.05843/) (2022)
- [Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunities](https://mlsanity.com/paper/2506.18019/) (2025)
- [LLM-Powered AI Agent Systems and Their Applications in Industry](https://mlsanity.com/paper/2505.16120/) (2025)
- [POGEMA: A Benchmark Platform for Cooperative Multi-Agent Pathfinding](https://mlsanity.com/paper/2407.14931/) (2024)
- [Learning Efficient Multi-Agent Cooperative Visual Exploration](https://mlsanity.com/paper/2110.05734/) (2021)
- [Pervasive AI for IoT applications: A Survey on Resource-efficient Distributed Artificial Intelligence](https://mlsanity.com/paper/2105.01798/) (2021)
- [A Survey on Deep Reinforcement Learning for Audio-Based Applications](https://mlsanity.com/paper/2101.00240/) (2021)
- [Distributed Reinforcement Learning for Robot Teams: A Review](https://mlsanity.com/paper/2204.03516/) (2022)
- [The Emergence of Wireless MAC Protocols with Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2108.07144/) (2021)
- [Multi-Agent Reinforcement Learning for Resources Allocation Optimization: A Survey](https://mlsanity.com/paper/2504.21048/) (2025)
- [Autonomy and Intelligence in the Computing Continuum: Challenges, Enablers, and Future Directions for Orchestration](https://mlsanity.com/paper/2205.01423/) (2022)
- [Effective and Stable Role-Based Multi-Agent Collaboration by Structural Information Principles](https://mlsanity.com/paper/2304.00755/) (2023)
- [Transformers in Reinforcement Learning: A Survey](https://mlsanity.com/paper/2307.05979/) (2023)
- [A Review of Safe Reinforcement Learning Methods for Modern Power Systems](https://mlsanity.com/paper/2407.00304/) (2024)
- [Multi-Agent Deep Reinforcement Learning for Dynamic Avatar Migration in AIoT-enabled Vehicular Metaverses with Trajectory Prediction](https://mlsanity.com/paper/2306.14683/) (2023)
- [On Limitation of Transformer for Learning HMMs](https://mlsanity.com/paper/2406.04089/) (2024)
- [Decentralized Transformers with Centralized Aggregation are Sample-Efficient Multi-Agent World Models](https://mlsanity.com/paper/2406.15836/) (2024)
- [Genes in Intelligent Agents](https://mlsanity.com/paper/2306.10225/) (2023)

## Abstract (arXiv)

Reinforcement learning (RL) algorithms have been around for decades and employed to solve various sequential decision-making problems. These algorithms however have faced great challenges when dealing with high-dimensional environments. The recent development of deep learning has enabled RL methods to drive optimal policies for sophisticated and capable agents, which can perform efficiently in these challenging environments. This paper addresses an important aspect of deep RL related to situations that require multiple agents to communicate and cooperate to solve complex tasks. A survey of different approaches to problems related to multi-agent deep RL (MADRL) is presented, including non-stationarity, partial observability, continuous state and action spaces, multi-agent training schemes, multi-agent transfer learning. The merits and demerits of the reviewed methods will be analyzed and discussed, with their corresponding applications explored. It is envisaged that this review provides insights about various MADRL methods and can lead to future development of more robust and highly useful multi-agent learning methods for solving real-world problems.