# Physics-informed Reinforcement Learning for Stochastic Reach-Avoid Analysis

arXiv: https://arxiv.org/abs/2608.17117 · HTML: https://mlsanity.com/paper/2608.17117/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 23 of its 42 references that are in the map.
- First posted on arXiv 17 Aug 2026.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 23 of 23)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Offline Reinforcement Learning with Implicit Q-Learning](https://mlsanity.com/paper/2110.06169/) (2021)
- [Hamilton-Jacobi Reachability: A Brief Overview and Recent Advances](https://mlsanity.com/paper/1709.07523/) (2017)
- [An Expert's Guide to Training Physics-informed Neural Networks](https://mlsanity.com/paper/2308.08468/) (2023)
- [Safety and Liveness Guarantees through Reach-Avoid Reinforcement Learning](https://mlsanity.com/paper/2112.12288/) (2021)
- [A Survey on Physics Informed Reinforcement Learning: Review and Open Problems](https://mlsanity.com/paper/2309.01909/) (2023)
- [Learning Control Policies for Stochastic Systems with Reach-avoid Guarantees](https://mlsanity.com/paper/2210.05308/) (2022)
- [Hamilton-Jacobi Reachability in Reinforcement Learning: A Survey](https://mlsanity.com/paper/2407.09645/) (2024)
- [How to Avoid Trivial Solutions in Physics-Informed Neural Networks](https://mlsanity.com/paper/2112.05620/) (2021)
- [Physics-Informed Neural Network Policy Iteration: Algorithms, Convergence, and Verification](https://mlsanity.com/paper/2402.10119/) (2024)
- [A Minimum Discounted Reward Hamilton-Jacobi Formulation for Computing Reachable Sets](https://mlsanity.com/paper/1809.00706/) (2018)
- [Bridging Physics-Informed Neural Networks with Reinforcement Learning: Hamilton-Jacobi-Bellman Proximal Policy Optimization (HJBPPO)](https://mlsanity.com/paper/2302.00237/) (2023)
- [Safe Control in the Presence of Stochastic Uncertainties](https://mlsanity.com/paper/2104.01259/) (2021)
- [Underapproximation of Reach-Avoid Sets for Discrete-Time Stochastic Systems via Lagrangian Methods](https://mlsanity.com/paper/1704.03555/) (2017)
- [Model-Free Stochastic Reachability Using Kernel Distribution Embeddings](https://mlsanity.com/paper/1908.00697/) (2019)
- [State-Based Confidence Bounds for Data-Driven Stochastic Reachability Using Hilbert Space Embeddings](https://mlsanity.com/paper/2010.08036/) (2020)
- [Sufficient and Necessary Barrier-like Conditions for Safety and Reach-avoid Verification of Stochastic Discrete-time Systems](https://mlsanity.com/paper/2408.15572/) (2024)
- [Generalizable Physics-Informed Learning for Stochastic Safety-Critical Systems](https://mlsanity.com/paper/2407.08868/) (2024)
- [Autonomous Drifting Based on Maximal Safety Probability Learning](https://mlsanity.com/paper/2409.03160/) (2024)
- [Probabilistic Reachability and Invariance Computation of Stochastic Systems using Linear Programming](https://mlsanity.com/paper/2211.07544/) (2022)
- [Physics-informed RL for Maximal Safety Probability Estimation](https://mlsanity.com/paper/2403.16391/) (2024)
- [Gradual Transition from Bellman Optimality Operator to Bellman Operator in Online Reinforcement Learning](https://mlsanity.com/paper/2506.05968/) (2025)

## Abstract (arXiv)

Stochastic reach-avoid analysis of controlled dynamical systems is an important tool for safety-critical control under uncertainty, in which the reach-avoid probability is characterized by a Hamilton-Jacobi partial differential equation (PDE). However, solving this PDE using conventional numerical methods becomes computationally intractable as the system dimension increases. Physics-informed neural networks (PINNs) may converge to inaccurate local minima when trained primarily through PDE-residual minimization. Reinforcement learning (RL) offers a scalable alternative, but its learned value functions may be inaccurate or inconsistent with the governing PDE. This paper proposes a physics-informed RL (PIRL) framework that combines the complementary strengths of PINNs and RL for stochastic reach-avoid analysis. We develop a scheduled PIRL algorithm in which temporal-difference actor-critic learning first guides the critic toward a meaningful approximation of the reach-avoid value function. PDE-residual and boundary-condition losses are then introduced progressively to enforce consistency with the governing PDE and its boundary conditions. The proposed method mitigates the failure modes of conventional PINN techniques while achieving accuracy comparable to that of successfully trained PINNs. The effectiveness of the proposed framework is demonstrated through two case studies.