# Enhancing MAP-Elites with Multiple Parallel Evolution Strategies

arXiv: https://arxiv.org/abs/2303.06137 · HTML: https://mlsanity.com/paper/2303.06137/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 47 references that are in the map.
- First posted on arXiv 10 Mar 2023.

## Builds on (most cited first, 25 of 25)

- [Curiosity-driven Exploration by Self-supervised Prediction](https://mlsanity.com/paper/1705.05363/) (2017)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [Evolution Strategies as a Scalable Alternative to Reinforcement Learning](https://mlsanity.com/paper/1703.03864/) (2017)
- [A Brief Survey of Deep Reinforcement Learning](https://mlsanity.com/paper/1708.05866/) (2017)
- [Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulation](https://mlsanity.com/paper/2106.13281/) (2021)
- [First return, then explore](https://mlsanity.com/paper/2004.12919/) (2020)
- [Robots that can adapt like animals](https://mlsanity.com/paper/1407.3501/) (2014)
- [Quality and Diversity Optimization: A Unifying Modular Framework](https://mlsanity.com/paper/1708.09251/) (2017)
- [Improving Exploration in Evolution Strategies for Deep Reinforcement Learning via a Population of Novelty-Seeking Agents](https://mlsanity.com/paper/1712.06560/) (2017)
- [Covariance Matrix Adaptation for the Rapid Illumination of Behavior Space](https://mlsanity.com/paper/1912.02400/) (2019)
- [Differentiable Quality Diversity](https://mlsanity.com/paper/2106.03894/) (2021)
- [Diversity Policy Gradient for Sample Efficient Quality-Diversity Optimization](https://mlsanity.com/paper/2006.08505/) (2020)
- [Discovering the Elite Hypervolume by Leveraging Interspecies Correlation](https://mlsanity.com/paper/1804.03906/) (2018)
- [Data-Efficient Design Exploration through Surrogate-Assisted Illumination](https://mlsanity.com/paper/1806.05865/) (2018)
- [Scaling MAP-Elites to Deep Neuroevolution](https://mlsanity.com/paper/2003.01825/) (2020)
- [Approximating Gradients for Differentiable Quality Diversity in Reinforcement Learning](https://mlsanity.com/paper/2202.03666/) (2022)
- [Reset-free Trial-and-Error Learning for Robot Damage Recovery](https://mlsanity.com/paper/1610.04213/) (2016)
- [Empirical analysis of PGA-MAP-Elites for Neuroevolution in Uncertain Domains](https://mlsanity.com/paper/2210.13156/) (2022)
- [Multi-Emitter MAP-Elites: Improving quality, diversity and convergence speed with heterogeneous sets of emitters](https://mlsanity.com/paper/2007.05352/) (2020)
- [Fast and stable MAP-Elites in noisy domains using deep grids](https://mlsanity.com/paper/2006.14253/) (2020)
- [Uncertain Quality-Diversity: Evaluation methodology and new methods for Quality-Diversity in Uncertain Domains](https://mlsanity.com/paper/2302.00463/) (2023)
- [Hierarchical Behavioral Repertoires with Unsupervised Descriptors](https://mlsanity.com/paper/1804.07127/) (2018)
- [Training Diverse High-Dimensional Controllers by Scaling Covariance Matrix Adaptation MAP-Annealing](https://mlsanity.com/paper/2210.02622/) (2022)
- [Learning to Walk Autonomously via Reset-Free Quality-Diversity](https://mlsanity.com/paper/2204.03655/) (2022)
- [Assessing Quality-Diversity Neuro-Evolution Algorithms Performance in Hard Exploration Problems](https://mlsanity.com/paper/2211.13742/) (2022)

## Built on by (most cited first, 2 of 2)

- [Embodied Co-Design for Rapidly Evolving Agents: Taxonomy, Frontiers, and Challenges](https://mlsanity.com/paper/2512.04770/) (2025)
- [Scaling Policy Gradient Quality-Diversity with Massive Parallelization via Behavioral Variations](https://mlsanity.com/paper/2501.18723/) (2025)

## Abstract (arXiv)

With the development of fast and massively parallel evaluations in many domains, Quality-Diversity (QD) algorithms, that already proved promising in a large range of applications, have seen their potential multiplied. However, we have yet to understand how to best use a large number of evaluations as using them for random variations alone is not always effective. High-dimensional search spaces are a typical situation where random variations struggle to effectively search. Another situation is uncertain settings where solutions can appear better than they truly are and naively evaluating more solutions might mislead QD algorithms. In this work, we propose MAP-Elites-Multi-ES (MEMES), a novel QD algorithm based on Evolution Strategies (ES) designed to exploit fast parallel evaluations more effectively. MEMES maintains multiple (up to 100) simultaneous ES processes, each with its own independent objective and reset mechanism designed for QD optimisation, all on just a single GPU. We show that MEMES outperforms both gradient-based and mutation-based QD algorithms on black-box optimisation and QD-Reinforcement-Learning tasks, demonstrating its benefit across domains. Additionally, our approach outperforms sampling-based QD methods in uncertain domains when given the same evaluation budget. Overall, MEMES generates reproducible solutions that are high-performing and diverse through large-scale ES optimisation on easily accessible hardware.