mlsanityData to 7 Oct 2026

Recovery RL: Safe Reinforcement Learning with Learned Recovery Zones

arXiv 2010.15920 ↗First posted 29 Oct 2020cs.LG cs.AI cs.ROcited by 156

Not on the map. Why?

Abstract (arXiv)

Safety remains a central obstacle preventing widespread use of RL in the real world: learning new tasks in uncertain environments requires extensive exploration, but safety requires limiting exploration. We propose Recovery RL, an algorithm which navigates this tradeoff by (1) leveraging offline data to learn about constraint violating zones before policy learning and (2) separating the goals of improving task performance and constraint satisfaction across two policies: a task policy that only optimizes the task reward and a recovery policy that guides the agent to safety when constraint violation is likely. We evaluate Recovery RL on 6 simulation domains, including two contact-rich manipulation tasks and an image-based navigation task, and an image-based obstacle avoidance task on a physical robot. We compare Recovery RL to 5 prior safe RL methods which jointly optimize for task performance and safety via constrained optimization or reward shaping and find that Recovery RL outperforms the next best prior method across all domains. Results suggest that Recovery RL trades off constraint violations and task successes 2 - 20 times more efficiently in simulation domains and 3 times more efficiently in physical experiments. See https://tinyurl.com/rl-recovery for videos and supplementary material.

Key facts. Cited by 156 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 21; 2022: 30; 2023: 25; 2024: 34; 2025: 20; 2026: 26.

156 papers.

Sort:

By area:

  1. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 450
  2. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 403
  3. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
  4. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  5. 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 169
  6. 2021ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learningcited by 98
  7. 2023The Safety Filter: A Unified View of Safety-Critical Control in Autonomous Systemscited by 77
  8. 2022Constrained Variational Policy Optimization for Safe Reinforcement Learningcited by 69
  9. 2025SafeVLA: Towards Safety Alignment of Vision-Language-Action Model via Constrained Learningcited by 69
  10. 2022Safe Reinforcement Learning by Imagining the Near Futurecited by 67
  11. 2024Safe Offline Reinforcement Learning with Feasibility-Guided Diffusion Modelcited by 62
  12. 2024Agile But Safe: Learning Collision-Free High-Speed Legged Locomotioncited by 53
38 more
  1. 2022Reachability Constrained Reinforcement Learningcited by 52
  2. 2021LazyDAgger: Reducing Context Switching in Interactive Imitation Learningcited by 50
  3. 2023State-wise Safe Reinforcement Learning: A Surveycited by 48
  4. 2021Safe Reinforcement Learning Using Advantage-Based Interventioncited by 45
  5. 2021Safe Reinforcement Learning Using Robust Control Barrier Functionscited by 44
  6. 2025Generalizing Safety Beyond Collision-Avoidance via Latent-Space Reachability Analysiscited by 44
  7. 2024A Survey of Constraint Formulations in Safe Reinforcement Learningcited by 42
  8. 2022On the Robustness of Safe Reinforcement Learning under Observational Perturbationscited by 38
  9. 2023Model-Based Runtime Monitoring with Interactive Imitation Learningcited by 37
  10. 2021Learning Barrier Certificates: Towards Safe Reinforcement Learning with Zero Training-time Violationscited by 36
  11. 2021Autonomous Reinforcement Learning via Subgoal Curriculacited by 32
  12. 2022Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizabilitycited by 32
  13. 2022Provably Safe Reinforcement Learning: Conceptual Analysis, Survey, and Benchmarkingcited by 31
  14. 2025Uncertainty-aware Latent Safety Filters for Avoiding Out-of-Distribution Failurescited by 30
  15. 2022Safe Reinforcement Learning for Legged Locomotioncited by 29
  16. 2022Fleet-DAgger: Interactive Robot Fleet Learning with Scalable Human Supervisioncited by 29
  17. 2022Towards Safe Reinforcement Learning with a Safety Editor Policycited by 28
  18. 2023Safe Exploration in Reinforcement Learning: A Generalized Formulation and Algorithmscited by 27
  19. 2022Sim-to-Lab-to-Real: Safe Reinforcement Learning with Shielding and Generalization Guaranteescited by 24
  20. 2022ISAACS: Iterative Soft Adversarial Actor-Critic for Safetycited by 24
  21. 2024IntervenGen: Interventional Data Generation for Robust and Data-Efficient Robot Imitation Learningcited by 24
  22. 2022L2C2: Locally Lipschitz Continuous Constraint towards Stable and Smooth Reinforcement Learningcited by 22
  23. 2024ActSafe: Active Exploration with Safety Constraints for Reinforcement Learningcited by 22
  24. 2022When to Ask for Help: Proactive Interventions in Autonomous Reinforcement Learningcited by 20
  25. 2023Trust Region-Based Safe Distributional Reinforcement Learning for Multiple Constraintscited by 20
  26. 2022Back to the Manifold: Recovering from Out-of-Distribution Statescited by 19
  27. 2021VisuoSpatial Foresight for Physical Sequential Fabric Manipulationcited by 18
  28. 2021SimNet: Enabling Robust Unknown Object Manipulation from Pure Synthetic Data via Stereocited by 18
  29. 2022Evaluating Model-free Reinforcement Learning toward Safety-critical Taskscited by 18
  30. 2023Constraint-Conditioned Policy Optimization for Versatile Safe Reinforcement Learningcited by 17
  31. 2022Safe Model-Based Reinforcement Learning with an Uncertainty-Aware Reachability Certificatecited by 16
  32. 2024Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomycited by 16
  33. 2022Safe Reinforcement Learning of Dynamic High-Dimensional Robotic Tasks: Navigation, Manipulation, Interactioncited by 15
  34. 2022Monte Carlo Augmented Actor-Critic for Sparse Reward Deep Reinforcement Learning from Suboptimal Demonstrationscited by 14
  35. 2023Guided Online Distillation: Promoting Safe Reinforcement Learning by Offline Demonstrationcited by 14
  36. 2021Safe Autonomous Racing via Approximate Reachability on Ego-visioncited by 13
  37. 2023A Survey of Demonstration Learningcited by 12
  38. 2024Constrained Reinforcement Learning with Smoothed Log Barrier Functioncited by 12

50 most cited of 156 All 156 (JSON)