# Physics-informed RL for Maximal Safety Probability Estimation

arXiv: https://arxiv.org/abs/2403.16391 · HTML: https://mlsanity.com/paper/2403.16391/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods.
- Builds on 17 of its 29 references that are in the map.
- First posted on arXiv 25 Mar 2024.

## Placed in

- Deep reinforcement learning methods / [Safe and constrained RL](https://mlsanity.com/deeprl/safe-and-constrained-rl/)

## Builds on (most cited first, 17 of 17)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next](https://mlsanity.com/paper/2201.05624/) (2022)
- [Physics-informed neural networks (PINNs) for fluid mechanics: A review](https://mlsanity.com/paper/2105.09506/) (2021)
- [Recovery RL: Safe Reinforcement Learning with Learned Recovery Zones](https://mlsanity.com/paper/2010.15920/) (2020)
- [Learning to be Safe: Deep RL with a Safety Critic](https://mlsanity.com/paper/2010.14603/) (2020)
- [Policy Gradient for Coherent Risk Measures](https://mlsanity.com/paper/1502.03919/) (2015)
- [Density Constrained Reinforcement Learning](https://mlsanity.com/paper/2106.12764/) (2021)
- [A Survey on Physics Informed Reinforcement Learning: Review and Open Problems](https://mlsanity.com/paper/2309.01909/) (2023)
- [Modern Monte Carlo Methods for Efficient Uncertainty Quantification and Propagation: A Survey](https://mlsanity.com/paper/2011.00680/) (2020)
- [Safe Learning and Optimization Techniques: Towards a Survey of the State of the Art](https://mlsanity.com/paper/2101.09505/) (2021)
- [A Risk-Sensitive Finite-Time Reachability Approach for Safety of Stochastic Dynamic Systems](https://mlsanity.com/paper/1902.11277/) (2019)
- [Barrier Functions Inspired Reward Shaping for Reinforcement Learning](https://mlsanity.com/paper/2403.01410/) (2024)
- [A Classification-based Approach for Approximate Reachability](https://mlsanity.com/paper/1803.03237/) (2018)
- [Approximate Q-Learning for Controlled Diffusion Processes and its Near Optimality](https://mlsanity.com/paper/2203.07499/) (2022)
- [Policy Gradients for Probabilistic Constrained Reinforcement Learning](https://mlsanity.com/paper/2210.00596/) (2022)
- [Myopically Verifiable Probabilistic Certificate for Long-term Safety](https://mlsanity.com/paper/2110.13380/) (2021)
- [A Generalizable Physics-informed Learning Framework for Risk Probability Estimation](https://mlsanity.com/paper/2305.06432/) (2023)

## Built on by (most cited first, 1 of 1)

- [Physics-informed Reinforcement Learning for Stochastic Reach-Avoid Analysis](https://mlsanity.com/paper/2608.17117/) (2026)

## Abstract (arXiv)

Accurate risk quantification and reachability analysis are crucial for safe control and learning, but sampling from rare events, risky states, or long-term trajectories can be prohibitively costly. Motivated by this, we study how to estimate the long-term safety probability of maximally safe actions without sufficient coverage of samples from risky states and long-term trajectories. The use of maximal safety probability in control and learning is expected to avoid conservative behaviors due to over-approximation of risk. Here, we first show that long-term safety probability, which is multiplicative in time, can be converted into additive costs and be solved using standard reinforcement learning methods. We then derive this probability as solutions of partial differential equations (PDEs) and propose Physics-Informed Reinforcement Learning (PIRL) algorithm. The proposed method can learn using sparse rewards because the physics constraints help propagate risk information through neighbors. This suggests that, for the purpose of extracting more information for efficient learning, physics constraints can serve as an alternative to reward shaping. The proposed method can also estimate long-term risk using short-term samples and deduce the risk of unsampled states. This feature is in stark contrast with the unconstrained deep RL that demands sufficient data coverage. These merits of the proposed method are demonstrated in numerical simulation.