mlsanityData to 14 Oct 2026

Cooperative Artificial Intelligence

arXiv 2202.09859 ↗First posted 20 Feb 2022cs.AI cs.GT cs.MAcited by 0

Not on the map. Why?

Abstract (arXiv)

In the future, artificial learning agents are likely to become increasingly widespread in our society. They will interact with both other learning agents and humans in a variety of complex settings including social dilemmas. We argue that there is a need for research on the intersection between game theory and artificial intelligence, with the goal of achieving cooperative artificial intelligence that can navigate social dilemmas well. We consider the problem of how an external agent can promote cooperation between artificial learners by distributing additional rewards and punishments based on observing the actions of the learners. We propose a rule for automatically learning how to create the right incentives by considering the anticipated parameter updates of each agent. Using this learning rule leads to cooperation with high social welfare in matrix games in which the agents would otherwise learn to defect with high probability. We show that the resulting cooperative outcome is stable in certain games even if the planning agent is turned off after a given number of episodes, while other games require ongoing intervention to maintain mutual cooperation. Finally, we reflect on what the goals of multi-agent reinforcement learning should be in the first place, and discuss the necessary building blocks towards the goal of building cooperative AI.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 80 references that are in the map. First posted on arXiv 20 Feb 2022.

Builds on

19 of 80 references.

References · 19

  1. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  2. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  3. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 575
  4. 2017A Brief Survey of Deep Reinforcement Learningcited by 483
  5. 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 366
  6. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 337
  7. 2017Learning with Opponent-Learning Awarenesscited by 222
  8. 2017Multi-agent Reinforcement Learning in Sequential Social Dilemmascited by 209
  9. 2015Multiagent Cooperation and Competition with Deep Reinforcement Learningcited by 184
  10. 2018A Survey and Critique of Multiagent Deep Reinforcement Learningcited by 177
  11. 2020Open Problems in Cooperative AIcited by 168
  12. 2017Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observabilitycited by 148
7 more
  1. 2017Maintaining cooperation in complex social dilemmas using deep reinforcement learningcited by 48
  2. 2017Cooperating with Machinescited by 39
  3. 2020Learning to Play No-Press Diplomacy with Best Response Policy Iterationcited by 26
  4. 2017Consequentialist conditional cooperation in social dilemmas with imperfect informationcited by 23
  5. 2015Bayesian Policy Reusecited by 8
  6. 2018Towards Cooperation in Sequential Prisoner's Dilemmas: a Deep Multiagent Reinforcement Learning Approachcited by 4
  7. 2013A Model of Human Cooperation in Social Dilemmascited by 2

Built on by

Not cited yet.