mlsanityData to 7 Oct 2026

Multi-Agent, Human-Agent and Beyond: A Survey on Cooperation in Social Dilemmas

arXiv 2402.17270 ↗First posted 27 Feb 2024cs.AI cs.GT cs.HCcited by 12

On the map: LLM agents and tool use → Social simulation and multi-agent debate

Abstract (arXiv)

The study of cooperation within social dilemmas has long been a fundamental topic across various disciplines, including computer science and social science. Recent advancements in Artificial Intelligence (AI) have significantly reshaped this field, offering fresh insights into understanding and enhancing cooperation. This survey examines three key areas at the intersection of AI and cooperation in social dilemmas. First, focusing on multi-agent cooperation, we review the intrinsic and external motivations that support cooperation among rational agents, and the methods employed to develop effective strategies against diverse opponents. Second, looking into human-agent cooperation, we discuss the current AI algorithms for cooperating with humans and the human biases towards AI agents. Third, we review the emergent field of leveraging AI agents to enhance cooperation among humans. We conclude by discussing future research avenues, such as using large language models, establishing unified theoretical frameworks, revisiting existing theories of human cooperation, and exploring multiple real-world applications.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 41 of its 113 references that are in the map. First posted on arXiv 27 Feb 2024.

Builds on

41 of 113 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Generative Agents: Interactive Simulacra of Human Behavior7 Apr

Other references · 40

  1. 2023A Survey of Large Language Modelscited by 2,537
  2. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 555
  3. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
  4. 2023Language to Rewards for Robotic Skill Synthesiscited by 307
  5. 2023Reward Design with Language Modelscited by 257
  6. 2023Building Cooperative Embodied Agents Modularly with Large Language Modelscited by 231
  7. 2017Learning with Opponent-Learning Awarenesscited by 218
  8. 2023Playing repeated games with Large Language Modelscited by 213
  9. 2017Multi-agent Reinforcement Learning in Sequential Social Dilemmascited by 206
  10. 2018A Survey and Critique of Multiagent Deep Reinforcement Learningcited by 173
  11. 2018Inequity aversion improves cooperation in intertemporal social dilemmascited by 106
  12. 2023Can Large Language Models Serve as Rational Players in Game Theory? A Systematic Analysiscited by 91
28 more
  1. 2023Language Instructed Reinforcement Learning for Human-AI Coordinationcited by 61
  2. 2020Learning to Incentivize Other Learning Agentscited by 55
  3. 2018Stable Opponent Shaping in Differentiable Gamescited by 49
  4. 2020Social diversity and social preferences in mixed-motive reinforcement learningcited by 49
  5. 2017Maintaining cooperation in complex social dilemmas using deep reinforcement learningcited by 48
  6. 2022Model-Free Opponent Shapingcited by 43
  7. 2017Cooperating with Machinescited by 39
  8. 2022COLA: Consistent Learning with Opponent-Learning Awarenesscited by 38
  9. 2014Efficient Model Learning for Human-Robot Collaborative Taskscited by 35
  10. 2018Evolving intrinsic motivations for altruistic behaviorcited by 30
  11. 2017Statistical physics of human cooperationcited by 27
  12. 2019Learning Reciprocity in Complex Sequential Social Dilemmascited by 24
  13. 2021A learning agent that acquires social norms from public sanctions in decentralized multi-agent settingscited by 24
  14. 2020Emergent Reciprocity and Team Formation from Randomized Uncertain Social Preferencescited by 21
  15. 2022Proximal Learning With Opponent-Learning Awarenesscited by 20
  16. 2020D3C: Reducing the Price of Anarchy in Multi-Agent Learningcited by 16
  17. 2020When to (or not to) trust intelligent machines: Insights from an evolutionary game theory analysis of trust in repeated gamescited by 14
  18. 2022The Importance of Credo in Multiagent Learningcited by 7
  19. 2023Meta-Value Learning: a General Framework for Learning with Learning Awarenesscited by 6
  20. 2022Exploring the Benefits of Teams in Multiagent Learningcited by 5
  21. 2021Exploring the Impact of Tunable Agents in Sequential Social Dilemmascited by 2
  22. 2022The art of compensation: how hybrid teams solve collective risk dilemmascited by 2
  23. 2023Learning to Participate through Trading of Reward Sharescited by 2
  24. 2023Learning Roles with Emergent Social Value Orientationscited by 2
  25. 2023Heterogeneous Social Value Orientation Leads to Meaningful Diversity in Sequential Social Dilemmascited by 2
  26. 2023Towards a Better Understanding of Learning with Multiagent Teamscited by 2
  27. 2022Tackling Asymmetric and Circular Sequential Social Dilemmas with Reinforcement Learning and Graph-based Tit-for-Tatcited by 1
  28. 2022Rethinking Safe Control in the Presence of Self-Seeking Humanscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 9; 2026: 3.

12 papers.

  1. 2025AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challengescited by 167
  2. 2025Creativity in LLM-based Multi-Agent Systems: A Surveycited by 23
  3. 2025Virtual Agent Economiescited by 17
  4. 2026Adaptive Collaboration with Humans: Metacognitive Policy Optimization for Multi-Agent LLMs with Continual Learningcited by 9
  5. 2025MIND: Towards Immersive Psychological Healing with Multi-agent Inner Dialoguecited by 6
  6. 2025Reputation as a Solution to Cooperation Collapse in LLM-based MASscited by 6
  7. 2025Multi-level Value Alignment in Agentic AI Systems: Survey and Perspectivescited by 4
  8. 2026Social physics in the age of artificial intelligencecited by 4
  9. 2025Psychological and behavioural responses in human-agent vs. human-human interactions: a systematic review and meta-analysiscited by 2
  10. 2025Scene-Aware Vectorized Memory Multi-Agent Framework with Cross-Modal Differentiated Quantization VLMs for Visually Impaired Assistancecited by 1
  11. 2025Single-Agent Scaling Fails Multi-Agent Intelligence: Towards Foundation Models with Native Multi-Agent Intelligencecited by 0
  12. 2026Learning Incentive Structures for Cooperative Resilience in Multi-Agent Systems under Social Dilemmascited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Overcoming the Machine Penalty with Imperfectly Fair AI Agents
  2. 2025Artificially intelligent agents in the social and behavioral sciences: A history and outlook
  3. 2026Toward Human-Centered Multi-Agent Systems: Integrating Cognition, Culture, Values, and Cooperation in AI Agents
  4. 2025The Coming Crisis of Multi-Agent Misalignment: AI Alignment Must Be a Dynamic and Social Process
  5. 2024Shall We Team Up: Exploring Spontaneous Cooperation of Competing LLM Agents
  6. 2025Super-additive Cooperation in Language Model Agents
  7. 2025When Trust Collides: Decoding Human-LLM Cooperation Dynamics through the Prisoner's Dilemma
  8. 2025Evolution of Cooperation in LLM-Agent Societies: A Preliminary Study Using Different Punishment Strategies