mlsanityData to 7 Oct 2026

Fully Decentralized Cooperative Multi-Agent Reinforcement Learning: A Survey

arXiv 2401.04934 ↗First posted 10 Jan 2024cs.MA cs.AI cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Cooperative multi-agent reinforcement learning is a powerful tool to solve many real-world cooperative tasks, but restrictions of real-world applications may require training the agents in a fully decentralized manner. Due to the lack of information about other agents, it is challenging to derive algorithms that can converge to the optimal joint policy in a fully decentralized setting. Thus, this research area has not been thoroughly studied. In this paper, we seek to systematically review the fully decentralized methods in two settings: maximizing a shared reward of all agents and maximizing the sum of individual rewards of all agents, and discuss open questions and future research directions.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 44 of its 62 references that are in the map. First posted on arXiv 10 Jan 2024.

Builds on

44 of 62 references.

References · 44

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015Trust Region Policy Optimizationcited by 2,503
  3. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  4. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  5. 2017Counterfactual Multi-Agent Policy Gradientscited by 733
  6. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  7. 2019The StarCraft Multi-Agent Challengecited by 589
  8. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 555
  9. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 372
  10. 2020Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?cited by 281
  11. 2020QPLEX: Duplex Dueling Multi-Agent Q-Learningcited by 267
  12. 2018Actor-Attention-Critic for Multi-Agent Reinforcement Learningcited by 259
32 more
  1. 2019Google Research Football: A Novel Reinforcement Learning Environmentcited by 257
  2. 2020Weighted QMIX: Expanding Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 197
  3. 2018Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agentscited by 189
  4. 2021Trust Region Policy Optimisation in Multi-Agent Reinforcement Learningcited by 180
  5. 2020FACMAC: Factored Multi-Agent Centralised Policy Gradientscited by 178
  6. 2017Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learningcited by 114
  7. 2021V-Learning -- A Simple, Efficient, Decentralized Algorithm for Multiagent RLcited by 111
  8. 2020Off-Policy Multi-Agent Decomposed Policy Gradientscited by 97
  9. 2019Deep Coordination Graphscited by 96
  10. 2020Qatten: A General Framework for Cooperative Multiagent Reinforcement Learningcited by 81
  11. 2020Deep Implicit Coordination Graphs for Multi-agent Reinforcement Learningcited by 58
  12. 2021Multi-Agent Reinforcement Learning for Active Voltage Control on Power Distribution Networkscited by 52
  13. 2021Provably Efficient Reinforcement Learning in Decentralized General-Sum Markov Gamescited by 42
  14. 2017Lenient Multi-Agent Deep Reinforcement Learningcited by 35
  15. 2021On Improving Model-Free Algorithms for Decentralized Multi-Agent Reinforcement Learningcited by 35
  16. 2015Decentralized Q-Learning for Stochastic Teams and Gamescited by 34
  17. 2021Gradient play in stochastic games: stationary points, convergence, and sample complexitycited by 32
  18. 2023Breaking the Curse of Multiagency: Provably Efficient Decentralized Multi-Agent RL with Function Approximationcited by 29
  19. 2023Breaking the Curse of Multiagents in a Large State Space: RL in Markov Games with Independent Linear Function Approximationcited by 26
  20. 2020Multi-Agent Trust Region Policy Optimizationcited by 19
  21. 2022More Centralized Training, Still Decentralized Execution: Multi-Agent Conditional Policy Factorizationcited by 19
  22. 2021Divergence-Regularized Multi-Agent Actor-Criticcited by 15
  23. 2022Iterated Reasoning with Mutual Information in Cooperative and Byzantine Decentralized Teamingcited by 14
  24. 2022On the convergence of policy gradient methods to Nash equilibria in general stochastic gamescited by 13
  25. 2019Decentralized Learning for Optimality in Stochastic Dynamic Teams and Games with Local Control and Global State Informationcited by 12
  26. 2023Learning Multi-Agent Intention-Aware Communication for Optimal Multi-Order Execution in Financecited by 7
  27. 2022Learning Stationary Nash Equilibrium Policies in n-Player Stochastic Games with Independent Chainscited by 6
  28. 2022MA2QL: A Minimalist Approach to Fully Decentralized Multi-Agent Reinforcement Learningcited by 5
  29. 2022Decentralized Policy Optimizationcited by 5
  30. 2023Best Possible Q-Learningcited by 4
  31. 2023Unsynchronized Decentralized Q-Learning: Two Timescale Analysis By Persistencecited by 3
  32. 2022Decentralized Policy Gradient for Nash Equilibria Learning of General-sum Stochastic Gamescited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

  1. 2026CRAFT: Grounded Multi-Agent Coordination Under Partial Informationcited by 3
  2. 2025Learning "Partner-Aware" Collaborators in Multi-Party Collaborationcited by 2
  3. 2025Fully Decentralized Cooperative Multi-Agent Reinforcement Learning is A Context Modeling Problemcited by 0
  4. 2026Merging model-based control with multi-agent reinforcement learning for multi-agent cooperative teaming strategiescited by 0
  5. 2026FedCritic-MIMO: Communication-Efficient Serverless Federated Critic Learning for Massive-MIMO Resource Control in Open and Disaggregated 6G RANscited by 0