# Hierarchical Behavioral Repertoires with Unsupervised Descriptors

arXiv: https://arxiv.org/abs/1804.07127 · HTML: https://mlsanity.com/paper/1804.07127/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Apr 2018.

## Built on by (most cited first, 12 of 12)

- [Unsupervised Behaviour Discovery with Quality-Diversity Optimisation](https://mlsanity.com/paper/2106.05648/) (2021)
- [The Quality-Diversity Transformer: Generating Behavior-Conditioned Trajectories with Decision Transformers](https://mlsanity.com/paper/2303.16207/) (2023)
- [BR-NS: an Archive-less Approach to Novelty Search](https://mlsanity.com/paper/2104.03936/) (2021)
- [Online Damage Recovery for Physical Robots with Hierarchical Quality-Diversity](https://mlsanity.com/paper/2210.09918/) (2022)
- [Hierarchical Quality-Diversity for Online Damage Recovery](https://mlsanity.com/paper/2204.05726/) (2022)
- [Evolving Populations of Diverse RL Agents with MAP-Elites](https://mlsanity.com/paper/2303.12803/) (2023)
- [Don't Bet on Luck Alone: Enhancing Behavioral Reproducibility of Quality-Diversity Solutions in Uncertain Domains](https://mlsanity.com/paper/2304.03672/) (2023)
- [Enhancing MAP-Elites with Multiple Parallel Evolution Strategies](https://mlsanity.com/paper/2303.06137/) (2023)
- [Generation Drawing/Grinding Trajectoy Based on Hierarchical CVAE](https://mlsanity.com/paper/2111.10954/) (2021)
- [Vector Quantized-Elites: Unsupervised and Problem-Agnostic Quality-Diversity Optimization](https://mlsanity.com/paper/2504.08057/) (2025)
- [DEI: Diversity in Evolutionary Inference for Quality-Diversity Search](https://mlsanity.com/paper/2605.27130/) (2026)
- [On the use of feature-maps and parameter control for improved quality-diversity meta-evolution](https://mlsanity.com/paper/2105.10317/) (2021)

## Abstract (arXiv)

Enabling artificial agents to automatically learn complex, versatile and high-performing behaviors is a long-lasting challenge. This paper presents a step in this direction with hierarchical behavioral repertoires that stack several behavioral repertoires to generate sophisticated behaviors. Each repertoire of this architecture uses the lower repertoires to create complex behaviors as sequences of simpler ones, while only the lowest repertoire directly controls the agent's movements. This paper also introduces a novel approach to automatically define behavioral descriptors thanks to an unsupervised neural network that organizes the produced high-level behaviors. The experiments show that the proposed architecture enables a robot to learn how to draw digits in an unsupervised manner after having learned to draw lines and arcs. Compared to traditional behavioral repertoires, the proposed architecture reduces the dimensionality of the optimization problems by orders of magnitude and provides behaviors with a twice better fitness. More importantly, it enables the transfer of knowledge between robots: a hierarchical repertoire evolved for a robotic arm to draw digits can be transferred to a humanoid robot by simply changing the lowest layer of the hierarchy. This enables the humanoid to draw digits although it has never been trained for this task.