# Unsupervised Behaviour Discovery with Quality-Diversity Optimisation

arXiv: https://arxiv.org/abs/2106.05648 · HTML: https://mlsanity.com/paper/2106.05648/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 54 references that are in the map.
- First posted on arXiv 10 Jun 2021.

## Builds on (most cited first, 17 of 17)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Illuminating search spaces by mapping elites](https://mlsanity.com/paper/1504.04909/) (2015)
- [First return, then explore](https://mlsanity.com/paper/2004.12919/) (2020)
- [Robots that can adapt like animals](https://mlsanity.com/paper/1407.3501/) (2014)
- [Quality and Diversity Optimization: A Unifying Modular Framework](https://mlsanity.com/paper/1708.09251/) (2017)
- [Using Centroidal Voronoi Tessellations to Scale Up the Multi-dimensional Archive of Phenotypic Elites Algorithm](https://mlsanity.com/paper/1610.05729/) (2016)
- [Autonomous skill discovery with Quality-Diversity and Unsupervised Descriptors](https://mlsanity.com/paper/1905.11874/) (2019)
- [Reset-free Trial-and-Error Learning for Robot Damage Recovery](https://mlsanity.com/paper/1610.04213/) (2016)
- [Unsupervised Learning of Goal Spaces for Intrinsically Motivated Goal Exploration](https://mlsanity.com/paper/1803.00781/) (2018)
- [Unsupervised Learning and Exploration of Reachable Outcome Space](https://mlsanity.com/paper/1909.05508/) (2019)
- [Transforming Exploratory Creativity with DeLeNoX](https://mlsanity.com/paper/2103.11715/) (2021)
- [Hierarchical Behavioral Repertoires with Unsupervised Descriptors](https://mlsanity.com/paper/1804.07127/) (2018)
- [Quality Diversity Through Surprise](https://mlsanity.com/paper/1807.02397/) (2018)
- [Data-Efficient Exploration, Optimization, and Modeling of Diverse Designs through Surrogate-Assisted Illumination](https://mlsanity.com/paper/1702.03713/) (2017)
- [Novelty Search makes Evolvability Inevitable](https://mlsanity.com/paper/2005.06224/) (2020)

## Built on by (most cited first, 30 of 30)

- [Neuroevolution is a Competitive Alternative to Reinforcement Learning for Skill Discovery](https://mlsanity.com/paper/2210.03516/) (2022)
- [Goals as Reward-Producing Programs](https://mlsanity.com/paper/2405.13242/) (2024)
- [Quality-Diversity Actor-Critic: Learning High-Performing and Diverse Behaviors via Value and Successor Features Critics](https://mlsanity.com/paper/2403.09930/) (2024)
- [Quality Diversity through Human Feedback: Towards Open-Ended Diversity-Driven Optimization](https://mlsanity.com/paper/2310.12103/) (2023)
- [Position: Foundation Agents as the Paradigm Shift for Decision Making](https://mlsanity.com/paper/2405.17009/) (2024)
- [Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Models](https://mlsanity.com/paper/2506.07121/) (2025)
- [Relevance-guided Unsupervised Discovery of Abilities with Quality-Diversity Algorithms](https://mlsanity.com/paper/2204.09828/) (2022)
- [Dominated Novelty Search: Rethinking Local Competition in Quality-Diversity](https://mlsanity.com/paper/2502.00593/) (2025)
- [Online Damage Recovery for Physical Robots with Hierarchical Quality-Diversity](https://mlsanity.com/paper/2210.09918/) (2022)
- [Discovering and Exploiting Sparse Rewards in a Learned Behavior Space](https://mlsanity.com/paper/2111.01919/) (2021)
- [Hierarchical Quality-Diversity for Online Damage Recovery](https://mlsanity.com/paper/2204.05726/) (2022)
- [Diversity from Human Feedback](https://mlsanity.com/paper/2310.06648/) (2023)
- [Leveraging Human Feedback to Evolve and Discover Novel Emergent Behaviors in Robot Swarms](https://mlsanity.com/paper/2305.16148/) (2023)
- [Generative Design through Quality-Diversity Data Synthesis and Language Models](https://mlsanity.com/paper/2405.09997/) (2024)
- [Enabling Adaptive Agent Training in Open-Ended Simulators by Targeting Diversity](https://mlsanity.com/paper/2411.04466/) (2024)
- [AutoQD: Automatic Discovery of Diverse Behaviors with Quality-Diversity Optimization](https://mlsanity.com/paper/2506.05634/) (2025)
- [Discovering Unsupervised Behaviours from Full-State Trajectories](https://mlsanity.com/paper/2211.15451/) (2022)
- [Quality-Diversity Optimisation on a Physical Robot Through Dynamics-Aware and Reset-Free Learning](https://mlsanity.com/paper/2304.12080/) (2023)
- [Discovering Quality-Diversity Algorithms via Meta-Black-Box Optimization](https://mlsanity.com/paper/2502.02190/) (2025)
- [Exploring Behavior Discovery Methods for Heterogeneous Swarms of Limited-Capability Robots](https://mlsanity.com/paper/2310.16941/) (2023)
- [Preparing for Black Swans: The Antifragility Imperative for Machine Learning](https://mlsanity.com/paper/2405.11397/) (2024)
- [Quality-Diversity with Limited Resources](https://mlsanity.com/paper/2406.03731/) (2024)
- [Vector Quantized-Elites: Unsupervised and Problem-Agnostic Quality-Diversity Optimization](https://mlsanity.com/paper/2504.08057/) (2025)
- [Periodic Skill Discovery](https://mlsanity.com/paper/2511.03187/) (2025)
- [Beyond Single-Model Optimization: Preserving Plasticity in Continual Reinforcement Learning](https://mlsanity.com/paper/2604.15414/) (2026)
- [Efficient Exploration using Model-Based Quality-Diversity with Gradients](https://mlsanity.com/paper/2211.12610/) (2022)
- [Complex System Exploration with Interactive Human Guidance](https://mlsanity.com/paper/2510.00794/) (2025)
- [Quality-Diversity Optimization as Multi-Objective Optimization](https://mlsanity.com/paper/2602.00478/) (2026)
- [Parameter-Efficient Neuroevolution for Diverse LLM Generation: Quality-Diversity Optimization via Prompt Embedding Evolution](https://mlsanity.com/paper/2605.09781/) (2026)
- [Continual Reinforcement Learning with Neuroevolution](https://mlsanity.com/paper/2610.01583/) (2026)

## Abstract (arXiv)

Quality-Diversity algorithms refer to a class of evolutionary algorithms designed to find a collection of diverse and high-performing solutions to a given problem. In robotics, such algorithms can be used for generating a collection of controllers covering most of the possible behaviours of a robot. To do so, these algorithms associate a behavioural descriptor to each of these behaviours. Each behavioural descriptor is used for estimating the novelty of one behaviour compared to the others. In most existing algorithms, the behavioural descriptor needs to be hand-coded, thus requiring prior knowledge about the task to solve. In this paper, we introduce: Autonomous Robots Realising their Abilities, an algorithm that uses a dimensionality reduction technique to automatically learn behavioural descriptors based on raw sensory data. The performance of this algorithm is assessed on three robotic tasks in simulation. The experimental results show that it performs similarly to traditional hand-coded approaches without the requirement to provide any hand-coded behavioural descriptor. In the collection of diverse and high-performing solutions, it also manages to find behaviours that are novel with respect to more features than its hand-coded baselines. Finally, we introduce a variant of the algorithm which is robust to the dimensionality of the behavioural descriptor space.