# A Minimum Discounted Reward Hamilton-Jacobi Formulation for Computing Reachable Sets

arXiv: https://arxiv.org/abs/1809.00706 · HTML: https://mlsanity.com/paper/1809.00706/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Sep 2018.

## Built on by (most cited first, 11 of 11)

- [Iterative Reachability Estimation for Safe Reinforcement Learning](https://mlsanity.com/paper/2309.13528/) (2023)
- [Hamilton-Jacobi Reachability in Reinforcement Learning: A Survey](https://mlsanity.com/paper/2407.09645/) (2024)
- [Human-AI Safety: A Descendant of Generative AI and Control Systems Safety](https://mlsanity.com/paper/2405.09794/) (2024)
- [Verifiable Safety Q-Filters via Hamilton-Jacobi Reachability and Multiplicative Q-Networks](https://mlsanity.com/paper/2506.15693/) (2025)
- [Certifying Hamilton-Jacobi Reachability Learned via Reinforcement Learning](https://mlsanity.com/paper/2602.16475/) (2026)
- [Probabilistic Safety Guarantee for Stochastic Control Systems Using Average Reward MDPs](https://mlsanity.com/paper/2511.08419/) (2025)
- [Unifying Hamilton-Jacobi Reachability and Reinforcement Learning](https://mlsanity.com/paper/2601.08050/) (2026)
- [Reachability-Preserving Bellman Operator for the Discounted Reach-Cost Value Function: Uniting Hamilton-Jacobi Reachability and Reinforcement Learning](https://mlsanity.com/paper/2607.07893/) (2026)
- [Physics-informed Reinforcement Learning for Stochastic Reach-Avoid Analysis](https://mlsanity.com/paper/2608.17117/) (2026)
- [VertexCBF: Improving Neural Control Barrier Functions via Vertex-Restricted Control Search](https://mlsanity.com/paper/2609.12831/) (2026)
- [Reachability-Aware Diffusion Policy Optimization](https://mlsanity.com/paper/2610.05969/) (2026)

## Abstract (arXiv)

We propose a novel formulation for approximating reachable sets through a minimum discounted reward optimal control problem. The formulation yields a continuous solution that can be obtained by solving a Hamilton-Jacobi equation. Furthermore, the numerical approximation to this solution can be obtained as the unique fixed-point to a contraction mapping. This allows for more efficient solution methods that could not be applied under traditional formulations for solving reachable sets. In addition, this formulation provides a link between reinforcement learning and learning reachable sets for systems with unknown dynamics, allowing algorithms from the former to be applied to the latter. We use two benchmark examples, double integrator, and pursuit-evasion games, to show the correctness of the formulation as well as its strengths in comparison to previous work.