mlsanityData to 7 Oct 2026

Hamilton-Jacobi Reachability in Reinforcement Learning: A Survey

arXiv 2407.09645 ↗First posted 12 Jul 2024eess.SY cs.LG cs.ROcited by 11

Not on the map. Why?

Abstract (arXiv)

Recent literature has proposed approaches that learn control policies with high performance while maintaining safety guarantees. Synthesizing Hamilton-Jacobi (HJ) reachable sets has become an effective tool for verifying safety and supervising the training of reinforcement learning-based control policies for complex, high-dimensional systems. Previously, HJ reachability was restricted to verifying low-dimensional dynamical systems primarily because the computational complexity of the dynamic programming approach it relied on grows exponentially with the number of system states. In recent years, a litany of proposed methods addresses this limitation by computing the reachability value function simultaneously with learning control policies to scale HJ reachability analysis while still maintaining a reliable estimate of the true reachable set. These HJ reachability approximations are used to improve the safety, and even reward performance, of learned control policies and can solve challenging tasks such as those with dynamic obstacles and/or with lidar-based or vision-based observations. In this survey paper, we review the recent developments in the field of HJ reachability estimation in reinforcement learning that would provide a foundational basis for further research into reliability in high-dimensional systems.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 61 of its 114 references that are in the map. First posted on arXiv 12 Jul 2024.

Builds on

61 of 114 references.

References · 61

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  2. 2021Convex optimizationcited by 2,479
  3. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  4. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  5. 2017Constrained Policy Optimizationcited by 928
  6. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  7. 2017Deep learning for universal linear embeddings of nonlinear dynamicscited by 491
  8. 2017Safe Reinforcement Learning via Shieldingcited by 427
  9. 2018Reward Constrained Policy Optimizationcited by 397
  10. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
  11. 2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 320
  12. 2020Towards Continual Reinforcement Learning: A Review and Perspectivescited by 268
38 more
  1. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 256
  2. 2020Projection-Based Constrained Policy Optimizationcited by 195
  3. 2017Hamilton-Jacobi Reachability: A Brief Overview and Recent Advancescited by 186
  4. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 169
  5. 2017Learning Koopman Invariant Subspaces for Dynamic Mode Decompositioncited by 156
  6. 2017A General Safety Framework for Learning-Based Control in Uncertain Robotic Systemscited by 150
  7. 2022Safe Control with Learned Certificates: A Survey of Neural Lyapunov, Barrier, and Contraction methodscited by 148
  8. 2022An optimal control perspective on diffusion-based generative modelingcited by 145
  9. 2020Neural Lyapunov Controlcited by 142
  10. 2019Lyapunov-based Safe Policy Optimization for Continuous Controlcited by 140
  11. 2020CRPO: A New Approach for Safe Reinforcement Learning with Convergence Guaranteecited by 128
  12. 2023A Definition of Continual Reinforcement Learningcited by 98
  13. 2023The Safety Filter: A Unified View of Safety-Critical Control in Autonomous Systemscited by 77
  14. 2022Reachability Constrained Reinforcement Learningcited by 52
  15. 2023State-wise Safe Reinforcement Learning: A Surveycited by 48
  16. 2021Robust Control Barrier-Value Functions for Safety-Critical Controlcited by 47
  17. 2021Safety and Liveness Guarantees through Reach-Avoid Reinforcement Learningcited by 43
  18. 2014Reach-Avoid Problems with Time-Varying Dynamics, Targets and Constraintscited by 41
  19. 2021Stabilizing Neural Control Using Self-Learned Almost Lyapunov Criticscited by 33
  20. 2023Iterative Reachability Estimation for Safe Reinforcement Learningcited by 33
  21. 2021Model-based Constrained Reinforcement Learning using Generalized Control Barrier Functioncited by 30
  22. 2021Feasible Actor-Critic: Constrained Reinforcement Learning for Ensuring Statewise Safetycited by 30
  23. 2020On Infusing Reachability-Based Safety Assurance within Planning Frameworks for Human-Robot Vehicle Interactionscited by 24
  24. 2022Sim-to-Lab-to-Real: Safe Reinforcement Learning with Shielding and Generalization Guaranteescited by 24
  25. 2022ISAACS: Iterative Soft Adversarial Actor-Critic for Safetycited by 24
  26. 2023Solving Stabilize-Avoid Optimal Control via Epigraph Form and Deep Reinforcement Learningcited by 24
  27. 2021Learn-to-Race: A Multimodal Control Environment for Autonomous Racingcited by 23
  28. 2022You Only Live Once: Single-Life Reinforcement Learningcited by 23
  29. 2016Algorithms for Overcoming the Curse of Dimensionality for Certain Hamilton-Jacobi Equations Arising in Control Theory and Elsewherecited by 21
  30. 2018PAC-Bayes Control: Learning Policies that Provably Generalize to Novel Environmentscited by 21
  31. 2018BaRC: Backward Reachability Curriculum for Robotic Reinforcement Learningcited by 21
  32. 2016Decomposition of Reachable Sets and Tubes for a Class of Nonlinear Systemscited by 16
  33. 2020Probably Approximately Correct Vision-Based Planning using Motion Primitivescited by 16
  34. 2022CUP: A Conservative Update Policy Algorithm for Safe Reinforcement Learningcited by 16
  35. 2022Safe Model-Based Reinforcement Learning with an Uncertainty-Aware Reachability Certificatecited by 16
  36. 2021Safe Autonomous Racing via Approximate Reachability on Ego-visioncited by 13
  37. 2019Adaptive dynamic programming for nonaffine nonlinear optimal control problem with state constraintscited by 12
  38. 2022Quantifying Safety of Learning-based Self-Driving Control Using Almost-Barrier Functionscited by 10

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 7.

11 papers.

  1. 2025Uncertainty-aware Latent Safety Filters for Avoiding Out-of-Distribution Failurescited by 30
  2. 2025Real-Time Out-of-Distribution Failure Prevention via Multi-Modal Reasoningcited by 10
  3. 2025One Subgoal at a Time: Zero-Shot Generalization to Arbitrary Linear Temporal Logic Requirements in Multi-Task Reinforcement Learningcited by 8
  4. 2026How Does the Lagrangian Guide Safe Reinforcement Learning through Diffusion Models?cited by 4
  5. 2025Verifiable Safety Q-Filters via Hamilton-Jacobi Reachability and Multiplicative Q-Networkscited by 3
  6. 2026Safe Control using Learned Safety Filters and Adaptive Conformal Inferencecited by 2
  7. 2026Beyond Hard Constraints: Budget-Conditioned Reachability For Safe Offline Reinforcement Learningcited by 1
  8. 2026Solving Parameter-Robust Avoid Problems with Unknown Feasibility using Reinforcement Learningcited by 0
  9. 2026COOL-MC: Verifying and Explaining RL Policies for Platelet Inventory Managementcited by 0
  10. 2026Safe Score Matching: Diffusion Policies with Hamilton-Jacobi Reachability for Online Safe Reinforcement Learningcited by 0
  11. 2026Reachability-Informed Reinforcement Learning for Multi-Impulse Interplanetary Transferscited by 0