# Reset-free Trial-and-Error Learning for Robot Damage Recovery

arXiv: https://arxiv.org/abs/1610.04213 · HTML: https://mlsanity.com/paper/1610.04213/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Oct 2016.

## Built on by (most cited first, 24 of 24)

- [OneTwoVLA: A Unified Vision-Language-Action Model with Adaptive Reasoning](https://mlsanity.com/paper/2505.11917/) (2025)
- [Unsupervised Behaviour Discovery with Quality-Diversity Optimisation](https://mlsanity.com/paper/2106.05648/) (2021)
- [A State-Distribution Matching Approach to Non-Episodic Reinforcement Learning](https://mlsanity.com/paper/2205.05212/) (2022)
- [Dynamics-Aware Quality-Diversity for Efficient Learning of Skill Repertoires](https://mlsanity.com/paper/2109.08522/) (2021)
- [Accelerated Quality-Diversity through Massive Parallelism](https://mlsanity.com/paper/2202.01258/) (2022)
- [Quality-Diversity Actor-Critic: Learning High-Performing and Diverse Behaviors via Value and Successor Features Critics](https://mlsanity.com/paper/2403.09930/) (2024)
- [Benchmarking Quality-Diversity Algorithms on Neuroevolution for Reinforcement Learning](https://mlsanity.com/paper/2211.02193/) (2022)
- [Relevance-guided Unsupervised Discovery of Abilities with Quality-Diversity Algorithms](https://mlsanity.com/paper/2204.09828/) (2022)
- [Learning to Walk Autonomously via Reset-Free Quality-Diversity](https://mlsanity.com/paper/2204.03655/) (2022)
- [Online Damage Recovery for Physical Robots with Hierarchical Quality-Diversity](https://mlsanity.com/paper/2210.09918/) (2022)
- [Large Language Models as In-context AI Generators for Quality-Diversity](https://mlsanity.com/paper/2404.15794/) (2024)
- [Synergizing Quality-Diversity with Descriptor-Conditioned Reinforcement Learning](https://mlsanity.com/paper/2401.08632/) (2023)
- [Hierarchical Quality-Diversity for Online Damage Recovery](https://mlsanity.com/paper/2204.05726/) (2022)
- [Severe Damage Recovery in Evolving Soft Robots through Differentiable Programming](https://mlsanity.com/paper/2206.06674/) (2022)
- [Don't Bet on Luck Alone: Enhancing Behavioral Reproducibility of Quality-Diversity Solutions in Uncertain Domains](https://mlsanity.com/paper/2304.03672/) (2023)
- [A Simple Approach to Continual Learning by Transferring Skill Parameters](https://mlsanity.com/paper/2110.10255/) (2021)
- [Efficient Learning of Locomotion Skills through the Discovery of Diverse Environmental Trajectory Generator Priors](https://mlsanity.com/paper/2210.04819/) (2022)
- [Discovering Unsupervised Behaviours from Full-State Trajectories](https://mlsanity.com/paper/2211.15451/) (2022)
- [Enhancing MAP-Elites with Multiple Parallel Evolution Strategies](https://mlsanity.com/paper/2303.06137/) (2023)
- [Vector Quantized-Elites: Unsupervised and Problem-Agnostic Quality-Diversity Optimization](https://mlsanity.com/paper/2504.08057/) (2025)
- [Damage Adaptation in Seconds for Architected Materials](https://mlsanity.com/paper/2606.17394/) (2026)
- [Unlock Reliable Skill Inference for Quadruped Adaptive Behavior by Skill Graph](https://mlsanity.com/paper/2311.06015/) (2023)
- [From Tabula Rasa to Emergent Abilities: Discovering Robot Skills via Real-World Unsupervised Quality-Diversity](https://mlsanity.com/paper/2508.19172/) (2025)
- [Adaptive GR(1) Specification Repair for Liveness-Preserving Shielding in Reinforcement Learning](https://mlsanity.com/paper/2511.02605/) (2025)

## Abstract (arXiv)

The high probability of hardware failures prevents many advanced robots (e.g., legged robots) from being confidently deployed in real-world situations (e.g., post-disaster rescue). Instead of attempting to diagnose the failures, robots could adapt by trial-and-error in order to be able to complete their tasks. In this situation, damage recovery can be seen as a Reinforcement Learning (RL) problem. However, the best RL algorithms for robotics require the robot and the environment to be reset to an initial state after each episode, that is, the robot is not learning autonomously. In addition, most of the RL methods for robotics do not scale well with complex robots (e.g., walking robots) and either cannot be used at all or take too long to converge to a solution (e.g., hours of learning). In this paper, we introduce a novel learning algorithm called "Reset-free Trial-and-Error" (RTE) that (1) breaks the complexity by pre-generating hundreds of possible behaviors with a dynamics simulator of the intact robot, and (2) allows complex robots to quickly recover from damage while completing their tasks and taking the environment into account. We evaluate our algorithm on a simulated wheeled robot, a simulated six-legged robot, and a real six-legged walking robot that are damaged in several ways (e.g., a missing leg, a shortened leg, faulty motor, etc.) and whose objective is to reach a sequence of targets in an arena. Our experiments show that the robots can recover most of their locomotion abilities in an environment with obstacles, and without any human intervention.