mlsanityData to 7 Oct 2026

Maintaining cooperation in complex social dilemmas using deep reinforcement learning

arXiv 1707.01068 ↗First posted 4 Jul 2017cs.AI cs.GT cs.MAcited by 48

Not on the map. Why?

Abstract (arXiv)

Social dilemmas are situations where individuals face a temptation to increase their payoffs at a cost to total welfare. Building artificially intelligent agents that achieve good outcomes in these situations is important because many real world interactions include a tension between selfish interests and the welfare of others. We show how to modify modern reinforcement learning methods to construct agents that act in ways that are simple to understand, nice (begin by cooperating), provokable (try to avoid being exploited), and forgiving (try to return to mutual cooperation). We show both theoretically and experimentally that such agents can maintain cooperation in Markov social dilemmas. Our construction does not require training methods beyond a modification of self-play, thus if an environment is such that good strategies can be constructed in the zero-sum case (eg. Atari) then we can construct agents that solve social dilemmas in this environment.

Key facts. Cited by 48 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Jul 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 10; 2022: 13; 2023: 6; 2024: 10; 2025: 5; 2026: 3.

48 papers.

Sort:

By area:

  1. 2021Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Potcited by 97
  2. 2021Maximum Entropy Population-Based Training for Zero-Shot Human-AI Coordinationcited by 69
  3. 2025Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agentscited by 66
  4. 2022Model-Free Opponent Shapingcited by 43
  5. 2021Deep Multiagent Reinforcement Learning: Challenges and Directionscited by 42
  6. 2022Melting Pot 2.0cited by 37
  7. 2023A Review of Cooperation in Multi-agent Learningcited by 27
  8. 2022AI for Global Climate Cooperation: Modeling Global Climate Negotiations, Agreements, and Long-Term Cooperation in RICE-Ncited by 22
  9. 2021Model-Based Opponent Modelingcited by 21
  10. 2022Proximal Learning With Opponent-Learning Awarenesscited by 20
  11. 2024A social path to human-like artificial intelligencecited by 20
  12. 2021Neural Recursive Belief States in Multi-Agent Reinforcement Learningcited by 18
36 more
  1. 2022Emergent Bartering Behaviour in Multi-Agent Reinforcement Learningcited by 18
  2. 2024Multi-Agent, Human-Agent and Beyond: A Survey on Cooperation in Social Dilemmascited by 12
  3. 2022Warmth and competence in human-agent cooperationcited by 11
  4. 2022The Good Shepherd: An Oracle Agent for Mechanism Designcited by 10
  5. 2024LOQA: Learning with Opponent Q-Learning Awarenesscited by 8
  6. 2021Normative Disagreement as a Challenge for Cooperative AIcited by 7
  7. 2022Artificial Intelligence and Spontaneous Collusioncited by 7
  8. 2024eQMARL: Entangled Quantum Multi-Agent Reinforcement Learning for Distributed Cooperation over Quantum Channelscited by 7
  9. 2024Advantage Alignment Algorithmscited by 7
  10. 2025SocialJax: An Evaluation Suite for Multi-agent Reinforcement Learning in Sequential Social Dilemmascited by 7
  11. 2024Learning to Balance Altruism and Self-interest Based on Empathy in Mixed-Motive Gamescited by 6
  12. 2021Modelling Cooperation in Network Games with Spatio-Temporal Complexitycited by 5
  13. 2023Multi-Agent Quantum Reinforcement Learning using Evolutionary Optimizationcited by 5
  14. 2025VS-Bench: Evaluating VLMs for Strategic Abilities in Multi-Agent Environmentscited by 5
  15. 2025Evaluating LLMs in Open-Source Gamescited by 5
  16. 2018Reinforcement Learning and Inverse Reinforcement Learning with System 1 and System 2cited by 4
  17. 2021Statistical discrimination in learning agentscited by 4
  18. 2023Mediated Multi-Agent Reinforcement Learningcited by 4
  19. 2021Balancing Rational and Other-Regarding Preferences in Cooperative-Competitive Environmentscited by 3
  20. 2022Stochastic Market Gamescited by 2
  21. 2023Learning to Participate through Trading of Reward Sharescited by 2
  22. 2023Heterogeneous Social Value Orientation Leads to Meaningful Diversity in Sequential Social Dilemmascited by 2
  23. 2024Reciprocal Reward Influence Encourages Cooperation From Self-Interested Agentscited by 2
  24. 2021Accumulating Risk Capital Through Investing in Cooperationcited by 1
  25. 2022Tackling Asymmetric and Circular Sequential Social Dilemmas with Reinforcement Learning and Graph-based Tit-for-Tatcited by 1
  26. 2022Intrinsic fluctuations of reinforcement learning promote cooperationcited by 1
  27. 2025The challenge of hidden gifts in multi-agent reinforcement learningcited by 1
  28. 2026Fairness over Equality: Correcting Social Incentives in Asymmetric Sequential Social Dilemmascited by 1
  29. 2022Cooperative Artificial Intelligencecited by 0
  30. 2022Decentralized scheduling through an adaptive, trading-based multi-agent systemcited by 0
  31. 2023AI4GCC -- Track 3: Consumption and the Challenges of Multi-Agent RLcited by 0
  32. 2024Architectural Influence on Variational Quantum Circuits in Multi-Agent Reinforcement Learning: Evolutionary Strategies for Optimizationcited by 0
  33. 2024Environment Complexity and Nash Equilibria in a Sequential Social Dilemmacited by 0
  34. 2024Managing multiple agents by automatically adjusting incentivescited by 0
  35. 2026Adaptive Punishment for Cooperation in Mixed-Motive Gamescited by 0
  36. 2026Calibrating Artificial Guilt: Neurally Grounded Reward Shaping for Prosocial Multi-Agent Reinforcement Learningcited by 0