mlsanityData to 7 Oct 2026

Multi-agent Reinforcement Learning in Sequential Social Dilemmas

arXiv 1702.03037 ↗First posted 10 Feb 2017cs.MA cs.AI cs.GTcited by 206

Not on the map. Why?

Cited by 34 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Matrix games like Prisoner's Dilemma have guided research on social dilemmas for decades. However, they necessarily treat the choice to cooperate or defect as an atomic action. In real-world social dilemmas these choices are temporally extended. Cooperativeness is a property that applies to policies, not elementary actions. We introduce sequential social dilemmas that share the mixed incentive structure of matrix game social dilemmas but also require agents to learn policies that implement their strategic intentions. We analyze the dynamics of policies learned by multiple self-interested independent learning agents, each using its own deep Q-network, on two Markov games we introduce here: 1. a fruit Gathering game and 2. a Wolfpack hunting game. We characterize how learned behavior in each domain changes as a function of environmental factors including resource abundance. Our experiments show how conflict can emerge from competition over shared resources and shed light on how the sequential nature of real world social dilemmas affects cooperation.

Key facts. Cited by 206 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Feb 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 2; 2019: 3; 2020: 0; 2021: 42; 2022: 32; 2023: 34; 2024: 28; 2025: 24; 2026: 41.

206 papers.

Sort:

By area:

  1. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 177
  2. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 169
  3. 2023Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordiacited by 119
  4. 2022SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learningcited by 118
  5. 2018Multi-Agent Generative Adversarial Imitation Learningcited by 99
  6. 2021Federated Reinforcement Learning: Techniques, Applications, and Open Challengescited by 79
  7. 2019Multi-Agent Adversarial Inverse Reinforcement Learningcited by 60
  8. 2021Deep Multiagent Reinforcement Learning: Challenges and Directionscited by 42
  9. 2024Werewolf Arena: A Case Study in LLM Evaluation via Social Deductioncited by 42
  10. 2021Sample-Efficient Learning of Stackelberg Equilibria in General-Sum Gamescited by 40
  11. 2021Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learningcited by 38
  12. 2022Melting Pot 2.0cited by 37
38 more
  1. 2021Discovering Diverse Multi-Agent Strategic Behavior via Reward Randomizationcited by 35
  2. 2021Deep reinforcement learning in medical imaging: A literature reviewcited by 34
  3. 2021Learning to Simulate Self-Driven Particles System with Coordinated Policy Optimizationcited by 33
  4. 2023MAESTRO: Open-Ended Environment Design for Multi-Agent Reinforcement Learningcited by 30
  5. 2021Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Compositioncited by 28
  6. 2023A Review of Cooperation in Multi-agent Learningcited by 27
  7. 2021Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent Learning Systemscited by 24
  8. 2021Collective eXplainable AI: Explaining Cooperative Strategies and Agent Contribution in Multiagent Reinforcement Learning with Shapley Valuescited by 24
  9. 2021A Cooperative-Competitive Multi-Agent Framework for Auto-bidding in Online Advertisingcited by 22
  10. 2022What is the Solution for State-Adversarial Multi-Agent Reinforcement Learning?cited by 21
  11. 2024A social path to human-like artificial intelligencecited by 20
  12. 2025LLM Economist: Large Population Models and Mechanism Design in Multi-Agent Generative Simulacracited by 20
  13. 2019Learner-aware Teaching: Inverse Reinforcement Learning with Preferences and Constraintscited by 18
  14. 2022Influencing Long-Term Behavior in Multiagent Reinforcement Learningcited by 18
  15. 2021Emergent Communication under Competitioncited by 17
  16. 2021How and Why to Manipulate Your Own Agent: On the Incentives of Users of Learning Agentscited by 17
  17. 2021Co-GAIL: Learning Diverse Strategies for Human-Robot Collaborationcited by 16
  18. 2021HAMMER: Multi-Level Coordination of Reinforcement Learning Agents via Learned Messagingcited by 12
  19. 2021Emergent Prosociality in Multi-Agent Games Through Giftingcited by 12
  20. 2024Multi-Agent, Human-Agent and Beyond: A Survey on Cooperation in Social Dilemmascited by 12
  21. 2022Warmth and competence in human-agent cooperationcited by 11
  22. 2022Distributed Reinforcement Learning for Robot Teams: A Reviewcited by 11
  23. 2024Can LLM-Augmented autonomous agents cooperate?, An evaluation of their cooperative capabilities through Melting Potcited by 11
  24. 2025Evaluating Generalization Capabilities of LLM-Based Agents in Mixed-Motive Scenarios Using Concordiacited by 11
  25. 2023Modeling Moral Choices in Social Dilemmas with Multi-Agent Reinforcement Learningcited by 10
  26. 2025Meta-Thinking in LLMs via Multi-Agent Reinforcement Learning: A Surveycited by 10
  27. 2026The Five Ws of Multi-Agent Communication: Who Talks to Whom, When, What, and Why -- A Survey from MARL to Emergent Language and LLMscited by 10
  28. 2022Provably Efficient Fictitious Play Policy Optimization for Zero-Sum Markov Games with Structured Transitionscited by 9
  29. 2023Information Design in Multi-Agent Reinforcement Learningcited by 9
  30. 2025Spontaneous Giving and Calculated Greed in Language Modelscited by 9
  31. 2021Joint Attention for Multi-Agent Coordination and Social Learningcited by 8
  32. 2021Cooperative Multi-Agent Fairness and Equivariant Policiescited by 8
  33. 2024Principal-Agent Reinforcement Learning: Orchestrating AI Agents with Contractscited by 8
  34. 2025An Outlook on the Opportunities and Challenges of Multi-Agent AI Systemscited by 8
  35. 2019Independent Generative Adversarial Self-Imitation Learning in Cooperative Multiagent Systemscited by 7
  36. 2021Birds of a Feather Flock Together: A Close Look at Cooperation Emergence via Multi-Agent RLcited by 7
  37. 2021Normative Disagreement as a Challenge for Cooperative AIcited by 7
  38. 2022The Importance of Credo in Multiagent Learningcited by 7

50 most cited of 206 All 206 (JSON)