# Certifying Hamilton-Jacobi Reachability Learned via Reinforcement Learning

arXiv: https://arxiv.org/abs/2602.16475 · HTML: https://mlsanity.com/paper/2602.16475/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 7 of its 18 references that are in the map.
- First posted on arXiv 18 Feb 2026.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 7 of 7)

- [Hamilton-Jacobi Reachability: A Brief Overview and Recent Advances](https://mlsanity.com/paper/1709.07523/) (2017)
- [DeepReach: A Deep Learning Approach to High-Dimensional Reachability](https://mlsanity.com/paper/2011.02082/) (2020)
- [Iterative Reachability Estimation for Safe Reinforcement Learning](https://mlsanity.com/paper/2309.13528/) (2023)
- [Decomposition of Reachable Sets and Tubes for a Class of Nonlinear Systems](https://mlsanity.com/paper/1611.00122/) (2016)
- [Distributional Hamilton-Jacobi-Bellman Equations for Continuous-Time Reinforcement Learning](https://mlsanity.com/paper/2205.12184/) (2022)
- [A Minimum Discounted Reward Hamilton-Jacobi Formulation for Computing Reachable Sets](https://mlsanity.com/paper/1809.00706/) (2018)
- [Certified Approximate Reachability (CARe): Formal Error Bounds on Deep Learning of Reachable Sets](https://mlsanity.com/paper/2503.23912/) (2025)

## Built on by (most cited first, 1 of 1)

- [Reachability-Preserving Bellman Operator for the Discounted Reach-Cost Value Function: Uniting Hamilton-Jacobi Reachability and Reinforcement Learning](https://mlsanity.com/paper/2607.07893/) (2026)

## Abstract (arXiv)

We present a framework to certify Hamilton--Jacobi (HJ) reachability learned by reinforcement learning (RL). Building on a discounted initial time travel-cost formulation that makes small-step RL value iteration provably equivalent to a forward Hamilton--Jacobi (HJ) equation with damping, we convert certified learning errors into calibrated inner/outer enclosures of strict backward reachable tube. The core device is an additive-offset identity: if Wλ solves the discounted travel-cost Hamilton--Jacobi--Bellman (HJB) equation, then W_varepsilon:=Wλ + varepsilon solves the same PDE with a constant offset λvarepsilon. This means that a uniform value error is exactly equal to a constant HJB offset. We establish this uniform value error via two routes: (A) a Bellman operator-residual bound, and (B) a HJB PDE-slack bound. Our framework preserves HJ-level safety semantics and is compatible with deep RL. We demonstrate the approach on a double-integrator system by formally certifying, via satisfiability modulo theories (SMT), a value function learned through reinforcement learning to induce provably correct inner and outer backward-reachable set enclosures over a compact region of interest.