# RecoveryChaining: Learning Local Recovery Policies for Robust Manipulation

arXiv: https://arxiv.org/abs/2410.13979 · HTML: https://mlsanity.com/paper/2410.13979/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 35 references that are in the map.
- First posted on arXiv 17 Oct 2024.

## Builds on (most cited first, 18 of 18)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [robosuite: A Modular Simulation Framework and Benchmark for Robot Learning](https://mlsanity.com/paper/2009.12293/) (2020)
- [Recovery RL: Safe Reinforcement Learning with Learned Recovery Zones](https://mlsanity.com/paper/2010.15920/) (2020)
- [A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms](https://mlsanity.com/paper/1907.03146/) (2019)
- [Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning](https://mlsanity.com/paper/2304.13653/) (2023)
- [Augmenting Reinforcement Learning with Behavior Primitives for Diverse Manipulation Tasks](https://mlsanity.com/paper/2110.03655/) (2021)
- [Learning to Grasp the Ungraspable with Emergent Extrinsic Dexterity](https://mlsanity.com/paper/2211.01500/) (2022)
- [Back to the Manifold: Recovering from Out-of-Distribution States](https://mlsanity.com/paper/2207.08673/) (2022)
- [Robustness via Retrying: Closed-Loop Robotic Manipulation with Self-Supervised Learning](https://mlsanity.com/paper/1810.03043/) (2018)
- [Learning to Compose Hierarchical Object-Centric Controllers for Robotic Manipulation](https://mlsanity.com/paper/2011.04627/) (2020)
- [Untangling Dense Non-Planar Knots by Learning Manipulation Features and Recovery Policies](https://mlsanity.com/paper/2107.08942/) (2021)
- [LS3: Latent Space Safe Sets for Long-Horizon Visuomotor Control of Sparse Reward Iterative Tasks](https://mlsanity.com/paper/2107.04775/) (2021)
- [Temporal Logic Imitation: Learning Plan-Satisficing Motion Policies from Demonstrations](https://mlsanity.com/paper/2206.04632/) (2022)
- [Efficient Recovery Learning using Model Predictive Meta-Reasoning](https://mlsanity.com/paper/2209.13605/) (2022)
- [Learning Skills to Patch Plans Based on Inaccurate Models](https://mlsanity.com/paper/2009.13732/) (2020)

## Built on by (most cited first, 9 of 9)

- [FLARE: A Failure-Aware Framework for Autonomous Correction and Recovery in Visual-Language Robotic Manipulation](https://mlsanity.com/paper/2608.26645/) (2026)
- [Failure Forecasting Boosts Robustness of Sim2Real Rhythmic Insertion Policies](https://mlsanity.com/paper/2507.06519/) (2025)
- [Learning Robust Execution in Robotic Manipulation with Agentic Reinforcement Learning](https://mlsanity.com/paper/2607.13818/) (2026)
- [Diffusing Trajectory Optimization Problems for Recovery During Multi-Finger Manipulation](https://mlsanity.com/paper/2510.07030/) (2025)
- [UniManip: General-Purpose Zero-Shot Robotic Manipulation with Agentic Operational Graph](https://mlsanity.com/paper/2602.13086/) (2026)
- [SafeLoop: Risk-Aware Rollback for Vision-Language-Action Manipulation](https://mlsanity.com/paper/2609.26313/) (2026)
- [Causeway: Restoring Task Accessibility for Instruction Switching in VLA Policies](https://mlsanity.com/paper/2609.30913/) (2026)
- [UniIntervene++: An Adaptive Intervention Agent for Efficient Real-World Reinforcement Learning](https://mlsanity.com/paper/2610.03620/) (2026)
- [Diagnosing and Recovering from Observation-Space Shift at Long-Horizon Skill Seams](https://mlsanity.com/paper/2610.10810/) (2026)

## Abstract (arXiv)

Model-based planners and controllers are commonly used to solve complex manipulation problems as they can efficiently optimize diverse objectives and generalize to long horizon tasks. However, they often fail during deployment due to noisy actuation, partial observability and imperfect models. To enable a robot to recover from such failures, we propose to use hierarchical reinforcement learning to learn a recovery policy. The recovery policy is triggered when a failure is detected based on sensory observations and seeks to take the robot to a state from which it can complete the task using the nominal model-based controllers. Our approach, called RecoveryChaining, uses a hybrid action space, where the model-based controllers are provided as additional nominal options which allows the recovery policy to decide how to recover, when to switch to a nominal controller and which controller to switch to even with sparse rewards. We evaluate our approach in three multi-step manipulation tasks with sparse rewards, where it learns significantly more robust recovery policies than those learned by baselines. We successfully transfer recovery policies learned in simulation to a physical robot to demonstrate the feasibility of sim-to-real transfer with our method.