mlsanityData to 14 Oct 2026

Unifying Hamilton-Jacobi Reachability and Reinforcement Learning

arXiv 2601.08050 ↗First posted 12 Jan 2026eess.SY cs.SYcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

We unify Hamilton-Jacobi (HJ) reachability and Reinforcement Learning (RL) through a proposed running cost formulation. We prove that the resultant travel-cost value function is the unique bounded viscosity solution of a time-dependent Hamilton-Jacobi Bellman (HJB) Partial Differential Equation (PDE) with zero terminal data, whose negative sublevel set equals the strict backward-reachable tube. Using a forward reparameterization and a contraction inducing Bellman update, we show that fixed points of small-step RL value iteration converge to the viscosity solution of the forward discounted HJB. Experiments on a classical benchmark validate this connection by demonstrating convergence of learned value functions toward semi-Lagrangian HJB solutions and by quantifying approximation error across the state space. These results empirically support the theoretical analysis, showing that the proposed framework preserves reachability-based safety semantics while remaining compatible with deep RL implementations.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 14 of its 33 references that are in the map. First posted on arXiv 12 Jan 2026.

Builds on

14 of 33 references.

References · 14

  1. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,238
  2. 2019Control Barrier Functions: Theory and Applicationscited by 372
  3. 2017Hamilton-Jacobi Reachability: A Brief Overview and Recent Advancescited by 201
  4. 2020DeepReach: A Deep Learning Approach to High-Dimensional Reachabilitycited by 91
  5. 2021Robust Control Barrier-Value Functions for Safety-Critical Controlcited by 51
  6. 2023Iterative Reachability Estimation for Safe Reinforcement Learningcited by 37
  7. 2016Algorithms for Overcoming the Curse of Dimensionality for Certain Hamilton-Jacobi Equations Arising in Control Theory and Elsewherecited by 24
  8. 2016Decomposition of Reachable Sets and Tubes for a Class of Nonlinear Systemscited by 18
  9. 2024Hamilton-Jacobi Reachability in Reinforcement Learning: A Surveycited by 16
  10. 2022Distributional Hamilton-Jacobi-Bellman Equations for Continuous-Time Reinforcement Learningcited by 12
  11. 2018A Minimum Discounted Reward Hamilton-Jacobi Formulation for Computing Reachable Setscited by 11
  12. 2018Inner-Approximating Reachable Sets for Polynomial Systems with Time-Varying Uncertaintiescited by 4
2 more
  1. 2019Backward Reachability for Polynomial Systems on A Finite Horizoncited by 2
  2. 2024Converse Barrier Certificates for Finite-time Safety Verification of Continuous-time Perturbed Deterministic Systemscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Certifying Hamilton-Jacobi Reachability Learned via Reinforcement Learning
  2. 2026Reachability-Preserving Bellman Operator for the Discounted Reach-Cost Value Function: Uniting Hamilton-Jacobi Reachability and Reinforcement Learning
  3. 2025A Temporal Difference Method for Stochastic Continuous Dynamics
  4. 2025Dual-Objective Reinforcement Learning with Novel Hamilton-Jacobi-Bellman Formulations
  5. 2025Continuous-Time Value Iteration for Multi-Agent Reinforcement Learning
  6. 2024On the stability of Lipschitz continuous control problems and its application to reinforcement learning
  7. 2024Pathwise Relaxed Optimal Control of Rough Differential Equations
  8. 2026Safe Score Matching: Diffusion Policies with Hamilton-Jacobi Reachability for Online Safe Reinforcement Learning