# Exploring the Benefits of Teams in Multiagent Learning

arXiv: https://arxiv.org/abs/2205.02328 · HTML: https://mlsanity.com/paper/2205.02328/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 30 references that are in the map.
- First posted on arXiv 4 May 2022.

## Builds on (most cited first, 15 of 15)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Emergent Tool Use From Multi-Agent Autocurricula](https://mlsanity.com/paper/1909.07528/) (2019)
- [Human-level performance in first-person multiplayer games with population-based deep reinforcement learning](https://mlsanity.com/paper/1807.01281/) (2018)
- [Multi-agent Reinforcement Learning in Sequential Social Dilemmas](https://mlsanity.com/paper/1702.03037/) (2017)
- [Inequity aversion improves cooperation in intertemporal social dilemmas](https://mlsanity.com/paper/1803.08884/) (2018)
- [Social diversity and social preferences in mixed-motive reinforcement learning](https://mlsanity.com/paper/2002.02325/) (2020)
- [Evolving intrinsic motivations for altruistic behavior](https://mlsanity.com/paper/1811.05931/) (2018)
- [Emergent Reciprocity and Team Formation from Randomized Uncertain Social Preferences](https://mlsanity.com/paper/2011.05373/) (2020)
- [Partner Selection for the Emergence of Cooperation in Multi-Agent Systems Using Reinforcement Learning](https://mlsanity.com/paper/1902.03185/) (2019)
- [Expected Value of Communication for Planning in Ad Hoc Teamwork](https://mlsanity.com/paper/2103.01171/) (2021)
- [Cooperation and Reputation Dynamics with Reinforcement Learning](https://mlsanity.com/paper/2102.07523/) (2021)
- [Natural Emergence of Heterogeneous Strategies in Artificially Intelligent Competitive Teams](https://mlsanity.com/paper/2007.03102/) (2020)
- [The Importance of Credo in Multiagent Learning](https://mlsanity.com/paper/2204.07471/) (2022)
- [Cooperative and Competitive Biases for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2101.06890/) (2021)
- [Grounding Artificial Intelligence in the Origins of Human Behavior](https://mlsanity.com/paper/2012.08564/) (2020)

## Built on by (most cited first, 5 of 5)

- [Multi-Agent, Human-Agent and Beyond: A Survey on Cooperation in Social Dilemmas](https://mlsanity.com/paper/2402.17270/) (2024)
- [DM2: Decentralized Multi-Agent Reinforcement Learning for Distribution Matching](https://mlsanity.com/paper/2206.00233/) (2022)
- [Presenting Multiagent Challenges in Team Sports Analytics](https://mlsanity.com/paper/2303.13660/) (2023)
- [Towards a Better Understanding of Learning with Multiagent Teams](https://mlsanity.com/paper/2306.16205/) (2023)
- [Learning to Learn Group Alignment: A Self-Tuning Credo Framework with Multiagent Teams](https://mlsanity.com/paper/2304.07337/) (2023)

## Abstract (arXiv)

For problems requiring cooperation, many multiagent systems implement solutions among either individual agents or across an entire population towards a common goal. Multiagent teams are primarily studied when in conflict; however, organizational psychology (OP) highlights the benefits of teams among human populations for learning how to coordinate and cooperate. In this paper, we propose a new model of multiagent teams for reinforcement learning (RL) agents inspired by OP and early work on teams in artificial intelligence. We validate our model using complex social dilemmas that are popular in recent multiagent RL and find that agents divided into teams develop cooperative pro-social policies despite incentives to not cooperate. Furthermore, agents are better able to coordinate and learn emergent roles within their teams and achieve higher rewards compared to when the interests of all agents are aligned.