# Natural Emergence of Heterogeneous Strategies in Artificially Intelligent Competitive Teams

arXiv: https://arxiv.org/abs/2007.03102 · HTML: https://mlsanity.com/paper/2007.03102/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jul 2020.

## Built on by (most cited first, 12 of 12)

- [Heterogeneous Multi-Robot Reinforcement Learning](https://mlsanity.com/paper/2301.07137/) (2023)
- [A General Learning Framework for Open Ad Hoc Teamwork Using Graph-based Policy Learning](https://mlsanity.com/paper/2210.05448/) (2022)
- [Concentration Network for Reinforcement Learning of Large-Scale Multi-Agent Systems](https://mlsanity.com/paper/2203.06416/) (2022)
- [Exploring the Benefits of Teams in Multiagent Learning](https://mlsanity.com/paper/2205.02328/) (2022)
- [Graph Attention Multi-Agent Fleet Autonomy for Advanced Air Mobility](https://mlsanity.com/paper/2302.07337/) (2023)
- [Knowledge-based Reasoning and Learning under Partial Observability in Ad Hoc Teamwork](https://mlsanity.com/paper/2306.00790/) (2023)
- [A Policy Resonance Approach to Solve the Problem of Responsibility Diffusion in Multiagent Reinforcement Learning](https://mlsanity.com/paper/2208.07753/) (2022)
- [Learning Heterogeneous Agent Cooperation via Multiagent League Training](https://mlsanity.com/paper/2211.11616/) (2022)
- [Events as Triggers for Behavioral Diversity in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2605.12388/) (2026)
- [Knowledge-based and Data-driven Reasoning and Learning for Ad Hoc Teamwork](https://mlsanity.com/paper/2208.11556/) (2022)
- [Prioritized League Reinforcement Learning for Large-Scale Heterogeneous Multiagent Systems](https://mlsanity.com/paper/2403.18057/) (2024)
- [On Swarm Leader Identification using Probing Policies](https://mlsanity.com/paper/2512.18146/) (2025)

## Abstract (arXiv)

Multi agent strategies in mixed cooperative-competitive environments can be hard to craft by hand because each agent needs to coordinate with its teammates while competing with its opponents. Learning based algorithms are appealing but many scenarios require heterogeneous agent behavior for the team's success and this increases the complexity of the learning algorithm. In this work, we develop a competitive multi agent environment called FortAttack in which two teams compete against each other. We corroborate that modeling agents with Graph Neural Networks and training them with Reinforcement Learning leads to the evolution of increasingly complex strategies for each team. We observe a natural emergence of heterogeneous behavior amongst homogeneous agents when such behavior can lead to the team's success. Such heterogeneous behavior from homogeneous agents is appealing because any agent can replace the role of another agent at test time. Finally, we propose ensemble training, in which we utilize the evolved opponent strategies to train a single policy for friendly agents.