# MAP-Elites with Descriptor-Conditioned Gradients and Archive Distillation into a Single Policy

arXiv: https://arxiv.org/abs/2303.03832 · HTML: https://mlsanity.com/paper/2303.03832/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Mar 2023.

## Built on by (most cited first, 20 of 20)

- [Generating Behaviorally Diverse Policies with Latent Diffusion Models](https://mlsanity.com/paper/2305.18738/) (2023)
- [Acquiring Diverse Skills using Curriculum Reinforcement Learning with Mixture of Experts](https://mlsanity.com/paper/2403.06966/) (2024)
- [Quality-Diversity Actor-Critic: Learning High-Performing and Diverse Behaviors via Value and Successor Features Critics](https://mlsanity.com/paper/2403.09930/) (2024)
- [Quality Diversity under Sparse Reward and Sparse Interaction: Application to Grasping in Robotics](https://mlsanity.com/paper/2308.05483/) (2023)
- [Synergizing Quality-Diversity with Descriptor-Conditioned Reinforcement Learning](https://mlsanity.com/paper/2401.08632/) (2023)
- [Embodied Co-Design for Rapidly Evolving Agents: Taxonomy, Frontiers, and Challenges](https://mlsanity.com/paper/2512.04770/) (2025)
- [Generative Design through Quality-Diversity Data Synthesis and Language Models](https://mlsanity.com/paper/2405.09997/) (2024)
- [Controller Distillation Reduces Fragile Brain-Body Co-Adaptation and Enables Migrations in MAP-Elites](https://mlsanity.com/paper/2504.06523/) (2025)
- [Towards Multi-Morphology Controllers with Diversity and Knowledge Distillation](https://mlsanity.com/paper/2404.14625/) (2024)
- [Discovering Quality-Diversity Algorithms via Meta-Black-Box Optimization](https://mlsanity.com/paper/2502.02190/) (2025)
- [Quality-Diversity with Limited Resources](https://mlsanity.com/paper/2406.03731/) (2024)
- [Quality Diversity for Robot Learning: Limitations and Future Directions](https://mlsanity.com/paper/2407.17515/) (2024)
- [Preference-Conditioned Gradient Variations for Multi-Objective Quality-Diversity](https://mlsanity.com/paper/2411.12433/) (2024)
- [Soft Quality-Diversity Optimization](https://mlsanity.com/paper/2512.00810/) (2025)
- [Scaling Policy Gradient Quality-Diversity with Massive Parallelization via Behavioral Variations](https://mlsanity.com/paper/2501.18723/) (2025)
- [Trajectory First: A Curriculum for Discovering Diverse Policies](https://mlsanity.com/paper/2506.01568/) (2025)
- [Parameter-Efficient Neuroevolution for Diverse LLM Generation: Quality-Diversity Optimization via Prompt Embedding Evolution](https://mlsanity.com/paper/2605.09781/) (2026)
- [Structure-Conditioned Actor-Critic Branches for Quality-Diversity Reinforcement Learning](https://mlsanity.com/paper/2606.08735/) (2026)
- [GAE: Graph-Augmented Evolution for Scientific Discovery via Reinforcement Optimization](https://mlsanity.com/paper/2607.10127/) (2026)
- [Behavioral Foundation Models for Quality Diversity](https://mlsanity.com/paper/2609.35615/) (2026)

## Abstract (arXiv)

Quality-Diversity algorithms, such as MAP-Elites, are a branch of Evolutionary Computation generating collections of diverse and high-performing solutions, that have been successfully applied to a variety of domains and particularly in evolutionary robotics. However, MAP-Elites performs a divergent search based on random mutations originating from Genetic Algorithms, and thus, is limited to evolving populations of low-dimensional solutions. PGA-MAP-Elites overcomes this limitation by integrating a gradient-based variation operator inspired by Deep Reinforcement Learning which enables the evolution of large neural networks. Although high-performing in many environments, PGA-MAP-Elites fails on several tasks where the convergent search of the gradient-based operator does not direct mutations towards archive-improving solutions. In this work, we present two contributions: (1) we enhance the Policy Gradient variation operator with a descriptor-conditioned critic that improves the archive across the entire descriptor space, (2) we exploit the actor-critic training to learn a descriptor-conditioned policy at no additional cost, distilling the knowledge of the archive into one single versatile policy that can execute the entire range of behaviors contained in the archive. Our algorithm, DCG-MAP-Elites improves the QD score over PGA-MAP-Elites by 82% on average, on a set of challenging locomotion tasks.