mlsanityData to 7 Oct 2026

Offline Goal-Conditioned Reinforcement Learning for Safety-Critical Tasks with Recovery Policy

arXiv 2403.01734 ↗First posted 4 Mar 2024cs.RO cs.AI cs.LGcited by 5

On the map: Offline RL and imitation learning from data → Diffusion and flow policies for offline RL

Abstract (arXiv)

Offline goal-conditioned reinforcement learning (GCRL) aims at solving goal-reaching tasks with sparse rewards from an offline dataset. While prior work has demonstrated various approaches for agents to learn near-optimal policies, these methods encounter limitations when dealing with diverse constraints in complex environments, such as safety constraints. Some of these approaches prioritize goal attainment without considering safety, while others excessively focus on safety at the expense of training efficiency. In this paper, we study the problem of constrained offline GCRL and propose a new method called Recovery-based Supervised Learning (RbSL) to accomplish safety-critical tasks with various goals. To evaluate the method performance, we build a benchmark based on the robot-fetching environment with a randomly positioned obstacle and use expert or random policies to generate an offline dataset. We compare RbSL with three offline GCRL algorithms and one offline safe RL algorithm. As a result, our method outperforms the existing state-of-the-art methods to a large extent. Furthermore, we validate the practicality and effectiveness of RbSL by deploying it on a real Panda manipulator. Code is available at https://github.com/Sunlighted/RbSL.git.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Diffusion and flow policies for offline RL, one of 13 directions in Offline RL and imitation learning from data. Builds on 36 of its 43 references that are in the map. First posted on arXiv 4 Mar 2024.

Builds on

36 of 43 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Swapped goal-conditioned offline reinforcement learning17 Feb
  2. 2022How Far I'll Go: Offline Goal-Conditioned Reinforcement Learning via f-Advantage Regression7 Jun
  3. 2022Rethinking Goal-conditioned Supervised Learning and Its Connection to Offline RL9 Feb

Other references · 33

  1. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  2. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  3. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  4. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  5. 2017Constrained Policy Optimizationcited by 928
  6. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
  7. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
  8. 2018Behavioral Cloning from Observationcited by 482
  9. 2017Safe Reinforcement Learning via Shieldingcited by 427
  10. 2018Reward Constrained Policy Optimizationcited by 397
  11. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
  12. 2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 320
21 more
  1. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 302
  2. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 256
  3. 2019Batch Policy Learning under Constraintscited by 242
  4. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problemscited by 228
  5. 2020Recovery RL: Safe Reinforcement Learning with Learned Recovery Zonescited by 156
  6. 2021Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skillscited by 125
  7. 2022Deep Whole-Body Control: Learning a Unified Policy for Manipulation and Locomotioncited by 98
  8. 2021Constraints Penalized Q-learning for Safe Offline Reinforcement Learningcited by 90
  9. 2022COptiDICE: Offline Constrained Reinforcement Learning via Stationary Distribution Correction Estimationcited by 69
  10. 2021Legged Robots that Keep on Learning: Fine-Tuning Locomotion Policies in the Real Worldcited by 64
  11. 2021panda-gym: Open-source goal-conditioned environments for robotic learningcited by 56
  12. 2022Graph Inverse Reinforcement Learning from Diverse Videoscited by 45
  13. 2017Hindsight policy gradientscited by 36
  14. 2022i-Sim2Real: Reinforcement Learning of Robotic Policies in Tight Human-Robot Interaction Loopscited by 30
  15. 2023Safe Offline Reinforcement Learning with Real-Time Budget Constraintscited by 26
  16. 2023Learning Goal-Conditioned Policies Offline with Self-Supervised Reward Shapingcited by 25
  17. 2022Generalization with Lossy Affordances: Leveraging Broad Offline Data for Learning Visuomotor Taskscited by 24
  18. 2023Distance Weighted Supervised Learning for Offline Interaction Datacited by 18
  19. 2019Learning to Manipulate Object Collections Using Grounded State Representationscited by 11
  20. 2022Learning Goal-Oriented Non-Prehensile Pushing in Cluttered Scenescited by 3
  21. 2022Safety Correction from Baseline: Towards the Risk-aware Policy in Robotics via Dual-agent Reinforcement Learningcited by 3

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 3.

5 papers.

  1. 2024AlignIQL: Policy Alignment in Implicit Q-Learning through Constrained Optimizationcited by 19
  2. 2025Graph-Assisted Stitching for Offline Hierarchical Reinforcement Learningcited by 12
  3. 2024FOSP: Fine-tuning Offline Safe Policy through World Modelscited by 3
  4. 2025Prompting Decision Transformers for Zero-Shot Reach-Avoid Policiescited by 0
  5. 2025Beyond the Proxy: Trajectory-Distilled Guidance for Offline GFlowNet Trainingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Provably Efficient Offline Goal-Conditioned Reinforcement Learning with General Function Approximation and Single-Policy Concentrability
  2. 2024Offline Safe Reinforcement Learning Using Trajectory Classification
  3. 2024OGBench: Benchmarking Offline Goal-Conditioned RL
  4. 2025Don't Trade Off Safety: Diffusion Regularization for Constrained Offline RL
  5. 2024Goal-Conditioned Decision Transformer for Multi-Goal Offline Reinforcement Learning
  6. 2023Constrained Decision Transformer for Offline Safe Reinforcement Learning
  7. 2022Hierarchical Planning Through Goal-Conditioned Offline Reinforcement Learning
  8. 2024Latent Safety-Constrained Policy Approach for Safe Offline Reinforcement Learning