# Mix-ME: Quality-Diversity for Multi-Agent Learning

arXiv: https://arxiv.org/abs/2311.01829 · HTML: https://mlsanity.com/paper/2311.01829/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 28 references that are in the map.
- First posted on arXiv 3 Nov 2023.

## Builds on (most cited first, 16 of 16)

- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Illuminating search spaces by mapping elites](https://mlsanity.com/paper/1504.04909/) (2015)
- [Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulation](https://mlsanity.com/paper/2106.13281/) (2021)
- [Robots that can adapt like animals](https://mlsanity.com/paper/1407.3501/) (2014)
- [Quality and Diversity Optimization: A Unifying Modular Framework](https://mlsanity.com/paper/1708.09251/) (2017)
- [Celebrating Diversity in Shared Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2106.02195/) (2021)
- [Discovering the Elite Hypervolume by Leveraging Interspecies Correlation](https://mlsanity.com/paper/1804.03906/) (2018)
- [Scaling MAP-Elites to Deep Neuroevolution](https://mlsanity.com/paper/2003.01825/) (2020)
- [MAESTRO: Open-Ended Environment Design for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2303.03376/) (2023)
- [Neuroevolution is a Competitive Alternative to Reinforcement Learning for Skill Discovery](https://mlsanity.com/paper/2210.03516/) (2022)
- [Ridge Rider: Finding Diverse Solutions by Following Eigenvectors of the Hessian](https://mlsanity.com/paper/2011.06505/) (2020)
- [Generalization in Cooperative Multi-Agent Systems](https://mlsanity.com/paper/2202.00104/) (2022)
- [Diverse Agents for Ad-Hoc Cooperation in Hanabi](https://mlsanity.com/paper/1907.03840/) (2019)
- [QDax: A Library for Quality-Diversity and Population-based Algorithms with Hardware Acceleration](https://mlsanity.com/paper/2308.03665/) (2023)
- [Accelerated Quality-Diversity through Massive Parallelism](https://mlsanity.com/paper/2202.01258/) (2022)
- [Hierarchical Quality-Diversity for Online Damage Recovery](https://mlsanity.com/paper/2204.05726/) (2022)

## Built on by (most cited first, 4 of 4)

- [Bridging Evolutionary Algorithms and Reinforcement Learning: A Comprehensive Survey on Hybrid Algorithms](https://mlsanity.com/paper/2401.11963/) (2024)
- [Controlling Behavioral Diversity in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2405.15054/) (2024)
- [Soft Quality-Diversity Optimization](https://mlsanity.com/paper/2512.00810/) (2025)
- [Robust Agents in Open-Ended Worlds](https://mlsanity.com/paper/2512.08139/) (2025)

## Abstract (arXiv)

In many real-world systems, such as adaptive robotics, achieving a single, optimised solution may be insufficient. Instead, a diverse set of high-performing solutions is often required to adapt to varying contexts and requirements. This is the realm of Quality-Diversity (QD), which aims to discover a collection of high-performing solutions, each with their own unique characteristics. QD methods have recently seen success in many domains, including robotics, where they have been used to discover damage-adaptive locomotion controllers. However, most existing work has focused on single-agent settings, despite many tasks of interest being multi-agent. To this end, we introduce Mix-ME, a novel multi-agent variant of the popular MAP-Elites algorithm that forms new solutions using a crossover-like operator by mixing together agents from different teams. We evaluate the proposed methods on a variety of partially observable continuous control tasks. Our evaluation shows that these multi-agent variants obtained by Mix-ME not only compete with single-agent baselines but also often outperform them in multi-agent settings under partial observability.