mlsanityData to 14 Oct 2026

Solving Minimum-Cost Reach Avoid using Reinforcement Learning

arXiv 2410.22600 ↗First posted 29 Oct 2024cs.LG cs.RO math.OCcited by 14

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

Current reinforcement-learning methods are unable to directly learn policies that solve the minimum cost reach-avoid problem to minimize cumulative costs subject to the constraints of reaching the goal and avoiding unsafe states, as the structure of this new optimization problem is incompatible with current methods. Instead, a surrogate problem is solved where all objectives are combined with a weighted sum. However, this surrogate objective results in suboptimal policies that do not directly minimize the cumulative cost. In this work, we propose RC-PPO, a reinforcement-learning-based method for solving the minimum-cost reach-avoid problem by using connections to Hamilton-Jacobi reachability. Empirical results demonstrate that RC-PPO learns policies with comparable goal-reaching rates to while achieving up to 57% lower cumulative costs compared to existing methods on a suite of minimum-cost reach-avoid benchmarks on the Mujoco simulator. The project page can be found at https://oswinso.xyz/rcppo.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 45 of its 77 references that are in the map. First posted on arXiv 29 Oct 2024.

Builds on

45 of 77 references.

References · 45

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2017Deep reinforcement learning from human preferencescited by 3,912
  4. 2021Convex optimizationcited by 2,515
  5. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  6. 2016Generative Adversarial Imitation Learningcited by 1,597
  7. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  8. 2017Hindsight Experience Replaycited by 1,136
  9. 2017Constrained Policy Optimizationcited by 983
  10. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 490
  11. 2018Reward Constrained Policy Optimizationcited by 417
  12. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 340
33 more
  1. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 316
  2. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 260
  3. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observationscited by 260
  4. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 253
  5. 2018Visual Reinforcement Learning with Imagined Goalscited by 251
  6. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 229
  7. 2017Hamilton-Jacobi Reachability: A Brief Overview and Recent Advancescited by 201
  8. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 158
  9. 2021Discovering and Achieving Goals via World Modelscited by 131
  10. 2021Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotationcited by 118
  11. 2018Unsupervised Control Through Non-Parametric Discriminative Rewardscited by 102
  12. 2020Learning with AMIGo: Adversarially Motivated Intrinsic Goalscited by 88
  13. 2018The Laplacian in RL: Learning Representations with Efficient Approximationscited by 86
  14. 2019Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal Generationcited by 83
  15. 2019Keeping Your Distance: Solving Sparse Reward Tasks Using Self-Balancing Shaped Rewardscited by 78
  16. 2018Variational Inverse Control with Events: A General Framework for Data-Driven Reward Definitioncited by 69
  17. 2022Saute RL: Almost Surely Safe Reinforcement Learning Using State Augmentationcited by 69
  18. 2022Reachability Constrained Reinforcement Learningcited by 59
  19. 2021Safety and Liveness Guarantees through Reach-Avoid Reinforcement Learningcited by 49
  20. 2019Task-Relevant Adversarial Imitation Learningcited by 45
  21. 2020Goal-Aware Prediction: Learning to Model What Matterscited by 45
  22. 2021Learning to Jump from Pixelscited by 45
  23. 2019Positive-Unlabeled Reward Learningcited by 37
  24. 2023Iterative Reachability Estimation for Safe Reinforcement Learningcited by 37
  25. 2018Few-Shot Goal Inference for Visuomotor Learning and Planningcited by 30
  26. 2023Solving Stabilize-Avoid Optimal Control via Epigraph Form and Deep Reinforcement Learningcited by 24
  27. 2020Semi-supervised reward learning for offline reinforcement learningcited by 23
  28. 2019Policy Continuation with Hindsight Inverse Dynamicscited by 22
  29. 2023Behavior Alignment via Reward Function Optimizationcited by 16
  30. 2020Accelerated Deep Reinforcement Learning Based Load Shedding for Emergency Voltage Controlcited by 6
  31. 2023Solving Richly Constrained Reinforcement Learning through State Augmentation and Reward Penaltiescited by 4
  32. 2024Active Disruption Avoidance and Trajectory Design for Tokamak Ramp-downs with Neural Differential Equations and Reinforcement Learningcited by 3
  33. 2021Optimal-Horizon Model-Predictive Control with Differential Dynamic Programmingcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 9.

14 papers.

Sort:

By area:

  1. 2025A Physics-Informed Machine Learning Framework for Safe and Optimal Control of Autonomous Systemscited by 12
  2. 2025Solving Multi-Agent Safe Optimal Control with Distributed Epigraph Form MARLcited by 6
  3. 2025Hierarchical Multi-Agent Reinforcement Learning with Control Barrier Functions for Safety-Critical Autonomous Systemscited by 5
  4. 2025Dual-Objective Reinforcement Learning with Novel Hamilton-Jacobi-Bellman Formulationscited by 3
  5. 2026Epigraph-Guided Flow Matching for Safe and Performant Offline Reinforcement Learningcited by 2
  6. 2026Value Functions for Temporal Logic: Optimal Policies and Safety Filterscited by 1
  7. 2025Prompting Decision Transformers for Zero-Shot Reach-Avoid Policiescited by 0
  8. 2026Solving Parameter-Robust Avoid Problems with Unknown Feasibility using Reinforcement Learningcited by 0
  9. 2026Stochastic Minimum-Cost Reach-Avoid Reinforcement Learningcited by 0
  10. 2026Robust Peak-cost Constrained Reinforcement Learningcited by 0
  11. 2026Learning Reach-Avoid Task with Reinforcement Learning: Vectorized Simulation and Benchmarkcited by 0
  12. 2026Humanoid Safe Stop via Learned Stoppability Valuecited by 0
2 more
  1. 2026LEAP-CBF: A Safety Filter for Uncertain Systems with Least-Effort Adversarial Potentialscited by 0
  2. 2026Safe Score Matching: Diffusion Policies with Hamilton-Jacobi Reachability for Online Safe Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026KLip-PPO: A per-sample KL perspective on PPO-Clip
  2. 2026Reachability-Aware Diffusion Policy Optimization
  3. 2025Proactive Constrained Policy Optimization with Preemptive Penalty
  4. 2025PPO in the Fisher-Rao geometry
  5. 2025Guided Policy Optimization under Partial Observability
  6. 2025Incentivizing Safer Actions in Policy Optimization for Constrained Reinforcement Learning
  7. 2024Proximal Policy Optimization with Adaptive Exploration
  8. 2024Beyond the Boundaries of Proximal Policy Optimization