mlsanityData to 7 Oct 2026

A Review of Cooperation in Multi-agent Learning

arXiv 2312.05162 ↗First posted 8 Dec 2023cs.MA cs.AI cs.GTcited by 27

Not on the map. Why?

Abstract (arXiv)

Cooperation in multi-agent learning (MAL) is a topic at the intersection of numerous disciplines, including game theory, economics, social sciences, and evolutionary biology. Research in this area aims to understand both how agents can coordinate effectively when goals are aligned and how they may cooperate in settings where gains from working together are possible but possibilities for conflict abound. In this paper we provide an overview of the fundamental concepts, problem settings and algorithms of multi-agent learning. This encompasses reinforcement learning, multi-agent sequential decision-making, challenges associated with multi-agent cooperation, and a comprehensive review of recent progress, along with an evaluation of relevant metrics. Finally we discuss open challenges in the field with the aim of inspiring new avenues for research.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 90 of its 146 references that are in the map. First posted on arXiv 8 Dec 2023.

Builds on

90 of 146 references.

References · 90

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2015Continuous control with deep reinforcement learningcited by 3,279
  4. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  5. 2015Trust Region Policy Optimizationcited by 2,503
  6. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  7. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  8. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 769
  9. 2017Counterfactual Multi-Agent Policy Gradientscited by 733
  10. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  11. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 561
  12. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 372
38 more
  1. 2016Learning Multiagent Communication with Backpropagationcited by 355
  2. 2017Population Based Training of Neural Networkscited by 351
  3. 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 345
  4. 2017StarCraft II: A New Challenge for Reinforcement Learningcited by 268
  5. 2020QPLEX: Duplex Dueling Multi-Agent Q-Learningcited by 267
  6. 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 260
  7. 2018Actor-Attention-Critic for Multi-Agent Reinforcement Learningcited by 259
  8. 2019Google Research Football: A Novel Reinforcement Learning Environmentcited by 257
  9. 2017Learning with Opponent-Learning Awarenesscited by 218
  10. 2020Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Taskscited by 207
  11. 2017Multi-agent Reinforcement Learning in Sequential Social Dilemmascited by 206
  12. 2018TarMAC: Targeted Multi-Agent Communicationcited by 201
  13. 2019The Hanabi Challenge: A New Frontier for AI Researchcited by 194
  14. 2020"Other-Play" for Zero-Shot Coordinationcited by 194
  15. 2018Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agentscited by 189
  16. 2021Trust Region Policy Optimisation in Multi-Agent Reinforcement Learningcited by 180
  17. 2018Learning Attentional Communication for Multi-Agent Cooperationcited by 170
  18. 2018Graph Convolutional Reinforcement Learningcited by 148
  19. 2021Collaborating with Humans without Human Datacited by 139
  20. 2023Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordiacited by 119
  21. 2017Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learningcited by 114
  22. 2018Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Taskscited by 114
  23. 2020Shared Experience Actor-Critic for Multi-Agent Reinforcement Learningcited by 114
  24. 2018Inequity aversion improves cooperation in intertemporal social dilemmascited by 106
  25. 2021Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Potcited by 97
  26. 2023ProAgent: Building Proactive Cooperative Agents with Large Language Modelscited by 97
  27. 2019Open-ended Learning in Symmetric Zero-sum Gamescited by 96
  28. 2019Deep Coordination Graphscited by 96
  29. 2019Learning to Schedule Communication in Multi-agent Reinforcement Learningcited by 89
  30. 2020Qatten: A General Framework for Cooperative Multiagent Reinforcement Learningcited by 81
  31. 2019Shapley Q-value: A Local Reward Approach to Solve Global Reward Gamescited by 71
  32. 2021Maximum Entropy Population-Based Training for Zero-Shot Human-AI Coordinationcited by 69
  33. 2017A multi-agent reinforcement learning model of common-pool resource appropriationcited by 68
  34. 2019Autocurricula and the Emergence of Innovation from Social Interaction: A Manifesto for Multi-Agent Intelligence Researchcited by 58
  35. 2019Emergent Coordination Through Competitioncited by 57
  36. 2020Learning to Incentivize Other Learning Agentscited by 55
  37. 2017VAIN: Attentional Multi-agent Predictive Modelingcited by 51
  38. 2020Multi-agent Reinforcement Learning for Networked System Controlcited by 51

50 most cited of 90 All 90 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 13; 2025: 7; 2026: 5.

27 papers.

Sort:

By area:

  1. 2024Cut the Crap: An Economical Communication Pipeline for LLM-based Multi-Agent Systemscited by 129
  2. 2023Cooperation on the Fly: Exploring Language Agents for Ad Hoc Teamwork in the Avalon Gamecited by 27
  3. 2024POGEMA: A Benchmark Platform for Cooperative Multi-Agent Pathfindingcited by 20
  4. 2025Virtual Agent Economiescited by 17
  5. 2026Colosseum: Auditing Collusion in Cooperative Multi-Agent Systemscited by 17
  6. 2025Group size effects and collective misalignment in LLM multi-agent systemscited by 16
  7. 2026Intelligent AI Delegationcited by 16
  8. 2026CoopEval: Benchmarking Cooperation-Sustaining Mechanisms and LLM Agents in Social Dilemmascited by 14
  9. 2025M3HF: Multi-agent Reinforcement Learning from Multi-phase Human Feedback of Mixed Qualitycited by 11
  10. 2024Towards Collaborative Intelligence: Propagating Intentions and Reasoning for Multi-Agent Coordination with Large Language Modelscited by 10
  11. 2024Aligning Individual and Collective Objectives in Multi-Agent Cooperationcited by 7
  12. 2024Heterogeneous Multi-Agent Reinforcement Learning for Zero-Shot Scalable Collaborationcited by 6
15 more
  1. 2024Learning to Balance Altruism and Self-interest Based on Empathy in Mixed-Motive Gamescited by 6
  2. 2025DSADF: Thinking Fast and Slow for Decision Makingcited by 6
  3. 2026Evaluating Collective Behaviour of Hundreds of LLM Agentscited by 6
  4. 2024Normative Modules: A Generative Agent Architecture for Learning Norms that Supports Multi-Agent Cooperationcited by 4
  5. 2024Enabling Multi-Robot Collaboration from Single-Human Guidancecited by 3
  6. 2024ROMAS: A Role-Based Multi-Agent System for Database monitoring and Planningcited by 3
  7. 2024Cooperation and Control in Delegation Gamescited by 2
  8. 2024Incentives to Build Houses, Trade Houses, or Trade House Building Skills in Simulated Worlds under Various Governing Systems or Institutions: Comparing Multi-agent Reinforcement Learning to Generative Agent-based Modelcited by 2
  9. 2023TAPE: Leveraging Agent Topology for Cooperative Multi-Agent Policy Gradientcited by 1
  10. 2024Enhancing Cooperation through Selective Interaction and Long-term Experiences in Multi-Agent Reinforcement Learningcited by 1
  11. 2025The AI in the Mirror: LLM Self-Recognition in an Iterated Public Goods Gamecited by 1
  12. 2024Environment Complexity and Nash Equilibria in a Sequential Social Dilemmacited by 0
  13. 2025Explaining Strategic Decisions in Multi-Agent Reinforcement Learning for Aerial Combat Tacticscited by 0
  14. 2025Understanding Action Effects through Instrumental Empowerment in Multi-Agent Reinforcement Learningcited by 0
  15. 2026In Trust We Survive: Emergent Trust Learningcited by 0