# Online Damage Recovery for Physical Robots with Hierarchical Quality-Diversity

arXiv: https://arxiv.org/abs/2210.09918 · HTML: https://mlsanity.com/paper/2210.09918/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 62 references that are in the map.
- First posted on arXiv 18 Oct 2022.

## Builds on (most cited first, 30 of 34)

- [Diversity is All You Need: Learning Skills without a Reward Function](https://mlsanity.com/paper/1802.06070/) (2018)
- [Solving Rubik's Cube with a Robot Hand](https://mlsanity.com/paper/1910.07113/) (2019)
- [Data-Efficient Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1805.08296/) (2018)
- [Illuminating search spaces by mapping elites](https://mlsanity.com/paper/1504.04909/) (2015)
- [Learning robust perceptive locomotion for quadrupedal robots in the wild](https://mlsanity.com/paper/2201.08117/) (2022)
- [First return, then explore](https://mlsanity.com/paper/2004.12919/) (2020)
- [Robots that can adapt like animals](https://mlsanity.com/paper/1407.3501/) (2014)
- [Quality and Diversity Optimization: A Unifying Modular Framework](https://mlsanity.com/paper/1708.09251/) (2017)
- [Quality-Diversity Optimization: a novel branch of stochastic optimization](https://mlsanity.com/paper/2012.04322/) (2020)
- [Autonomous skill discovery with Quality-Diversity and Unsupervised Descriptors](https://mlsanity.com/paper/1905.11874/) (2019)
- [Hierarchical Skills for Efficient Exploration](https://mlsanity.com/paper/2110.10809/) (2021)
- [A survey on policy search algorithms for learning robot controllers in a handful of trials](https://mlsanity.com/paper/1807.02303/) (2018)
- [Unsupervised Behaviour Discovery with Quality-Diversity Optimisation](https://mlsanity.com/paper/2106.05648/) (2021)
- [Generating and Blending Game Levels via Quality-Diversity in the Latent Space of a Variational Autoencoder](https://mlsanity.com/paper/2102.12463/) (2021)
- [Reset-free Trial-and-Error Learning for Robot Damage Recovery](https://mlsanity.com/paper/1610.04213/) (2016)
- [Unsupervised Learning and Exploration of Reachable Outcome Space](https://mlsanity.com/paper/1909.05508/) (2019)
- [Quality Diversity for Multi-task Optimization](https://mlsanity.com/paper/2003.04407/) (2020)
- [From Pixels to Legs: Hierarchical Learning of Quadruped Locomotion](https://mlsanity.com/paper/2011.11722/) (2020)
- [A Quality Diversity Approach to Automatically Generating Human-Robot Interaction Scenarios in Shared Autonomy](https://mlsanity.com/paper/2012.04283/) (2020)
- [Curiosity Driven Exploration of Learned Disentangled Goal Spaces](https://mlsanity.com/paper/1807.01521/) (2018)
- [Finding Game Levels with the Right Difficulty in a Few Trials through Intelligent Trial-and-Error](https://mlsanity.com/paper/2005.07677/) (2020)
- [Illuminating the Space of Beatable Lode Runner Levels Produced By Various Generative Adversarial Networks](https://mlsanity.com/paper/2101.07868/) (2021)
- [From exploration to control: learning object manipulation skills through novelty search and local adaptation](https://mlsanity.com/paper/1901.00811/) (2019)
- [Hierarchically Organized Latent Modules for Exploratory Search in Morphogenetic Systems](https://mlsanity.com/paper/2007.01195/) (2020)
- [Hierarchical Behavioral Repertoires with Unsupervised Descriptors](https://mlsanity.com/paper/1804.07127/) (2018)
- [Planning in Learned Latent Action Spaces for Generalizable Legged Locomotion](https://mlsanity.com/paper/2008.11867/) (2020)
- [Adaptive Prior Selection for Repertoire-based Online Adaptation in Robotics](https://mlsanity.com/paper/1907.07029/) (2019)
- [MAP-Elites enables Powerful Stepping Stones and Diversity for Modular Robotics](https://mlsanity.com/paper/2012.04375/) (2020)
- [BR-NS: an Archive-less Approach to Novelty Search](https://mlsanity.com/paper/2104.03936/) (2021)
- [Adaptive CLF-MPC With Application To Quadrupedal Robots](https://mlsanity.com/paper/2112.04536/) (2021)

## Built on by (most cited first, 7 of 7)

- [Improving the Data Efficiency of Multi-Objective Quality-Diversity through Gradient Assistance and Crowding Exploration](https://mlsanity.com/paper/2302.12668/) (2023)
- [Adaptive Control Strategy for Quadruped Robots in Actuator Degradation Scenarios](https://mlsanity.com/paper/2312.17606/) (2023)
- [Multi-Objective Quality-Diversity in Unstructured and Unbounded Spaces](https://mlsanity.com/paper/2504.03715/) (2025)
- [Moving On, Even When You're Broken: Fail-Active Trajectory Generation via Diffusion Policies Conditioned on Embodiment and Task](https://mlsanity.com/paper/2602.02895/) (2026)
- [Self-evolving Embodied AI](https://mlsanity.com/paper/2602.04411/) (2026)
- [Preference-Conditioned Gradient Variations for Multi-Objective Quality-Diversity](https://mlsanity.com/paper/2411.12433/) (2024)
- [Diverse and Adaptable Arm Coordination for Octopus-Crawling via Diffusion-Based Uncertainty-Aware Optimization](https://mlsanity.com/paper/2609.21138/) (2026)

## Abstract (arXiv)

In real-world environments, robots need to be resilient to damages and robust to unforeseen scenarios. Quality-Diversity (QD) algorithms have been successfully used to make robots adapt to damages in seconds by leveraging a diverse set of learned skills. A high diversity of skills increases the chances of a robot to succeed at overcoming new situations since there are more potential alternatives to solve a new task.However, finding and storing a large behavioural diversity of multiple skills often leads to an increase in computational complexity. Furthermore, robot planning in a large skill space is an additional challenge that arises with an increased number of skills. Hierarchical structures can help reducing this search and storage complexity by breaking down skills into primitive skills. In this paper, we introduce the Hierarchical Trial and Error algorithm, which uses a hierarchical behavioural repertoire to learn diverse skills and leverages them to make the robot adapt quickly in the physical world. We show that the hierarchical decomposition of skills enables the robot to learn more complex behaviours while keeping the learning of the repertoire tractable. Experiments with a hexapod robot show that our method solves a maze navigation tasks with 20% less actions in simulation, and 43% less actions in the physical world, for the most challenging scenarios than the best baselines while having 78% less complete failures.