# Empirical analysis of PGA-MAP-Elites for Neuroevolution in Uncertain Domains

arXiv: https://arxiv.org/abs/2210.13156 · HTML: https://mlsanity.com/paper/2210.13156/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Oct 2022.

## Built on by (most cited first, 15 of 15)

- [Bridging Evolutionary Algorithms and Reinforcement Learning: A Comprehensive Survey on Hybrid Algorithms](https://mlsanity.com/paper/2401.11963/) (2024)
- [Proximal Policy Gradient Arborescence for Quality Diversity Reinforcement Learning](https://mlsanity.com/paper/2305.13795/) (2023)
- [QDax: A Library for Quality-Diversity and Population-based Algorithms with Hardware Acceleration](https://mlsanity.com/paper/2308.03665/) (2023)
- [The Quality-Diversity Transformer: Generating Behavior-Conditioned Trajectories with Decision Transformers](https://mlsanity.com/paper/2303.16207/) (2023)
- [Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Models](https://mlsanity.com/paper/2506.07121/) (2025)
- [Understanding the Synergies between Quality-Diversity and Deep Reinforcement Learning](https://mlsanity.com/paper/2303.06164/) (2023)
- [Synergizing Quality-Diversity with Descriptor-Conditioned Reinforcement Learning](https://mlsanity.com/paper/2401.08632/) (2023)
- [Assessing Quality-Diversity Neuro-Evolution Algorithms Performance in Hard Exploration Problems](https://mlsanity.com/paper/2211.13742/) (2022)
- [Don't Bet on Luck Alone: Enhancing Behavioral Reproducibility of Quality-Diversity Solutions in Uncertain Domains](https://mlsanity.com/paper/2304.03672/) (2023)
- [Efficient Quality-Diversity Optimization through Diverse Quality Species](https://mlsanity.com/paper/2304.07425/) (2023)
- [Improving the Data Efficiency of Multi-Objective Quality-Diversity through Gradient Assistance and Crowding Exploration](https://mlsanity.com/paper/2302.12668/) (2023)
- [Enhancing MAP-Elites with Multiple Parallel Evolution Strategies](https://mlsanity.com/paper/2303.06137/) (2023)
- [Quality-Diversity with Limited Resources](https://mlsanity.com/paper/2406.03731/) (2024)
- [Quality Diversity for Robot Learning: Limitations and Future Directions](https://mlsanity.com/paper/2407.17515/) (2024)
- [Preference-Conditioned Gradient Variations for Multi-Objective Quality-Diversity](https://mlsanity.com/paper/2411.12433/) (2024)

## Abstract (arXiv)

Quality-Diversity algorithms, among which MAP-Elites, have emerged as powerful alternatives to performance-only optimisation approaches as they enable generating collections of diverse and high-performing solutions to an optimisation problem. However, they are often limited to low-dimensional search spaces and deterministic environments. The recently introduced Policy Gradient Assisted MAP-Elites (PGA-MAP-Elites) algorithm overcomes this limitation by pairing the traditional Genetic operator of MAP-Elites with a gradient-based operator inspired by Deep Reinforcement Learning. This new operator guides mutations toward high-performing solutions using policy-gradients. In this work, we propose an in-depth study of PGA-MAP-Elites. We demonstrate the benefits of policy-gradients on the performance of the algorithm and the reproducibility of the generated solutions when considering uncertain domains. We first prove that PGA-MAP-Elites is highly performant in both deterministic and uncertain high-dimensional environments, decorrelating the two challenges it tackles. Secondly, we show that in addition to outperforming all the considered baselines, the collections of solutions generated by PGA-MAP-Elites are highly reproducible in uncertain environments, approaching the reproducibility of solutions found by Quality-Diversity approaches built specifically for uncertain applications. Finally, we propose an ablation and in-depth analysis of the dynamic of the policy-gradients-based variation. We demonstrate that the policy-gradient variation operator is determinant to guarantee the performance of PGA-MAP-Elites but is only essential during the early stage of the process, where it finds high-performing regions of the search space.