# A Deep Reinforcement Learning Approach to Rare Event Estimation

arXiv: https://arxiv.org/abs/2211.12470 · HTML: https://mlsanity.com/paper/2211.12470/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 36 references that are in the map.
- First posted on arXiv 22 Nov 2022.

## Builds on (most cited first, 11 of 11)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Adaptive Stress Testing for Autonomous Vehicles](https://mlsanity.com/paper/1902.01909/) (2019)
- [A Survey of Algorithms for Black-Box Safety Validation of Cyber-Physical Systems](https://mlsanity.com/paper/2005.02979/) (2020)
- [Neural Bridge Sampling for Evaluating Safety-Critical Autonomous Systems](https://mlsanity.com/paper/2008.10581/) (2020)
- [Adaptive Stress Testing with Reward Augmentation for Autonomous Vehicle Validation](https://mlsanity.com/paper/1908.01046/) (2019)
- [Training Adversarial Agents to Exploit Weaknesses in Deep Control Policies](https://mlsanity.com/paper/2002.12078/) (2020)
- [A reinforcement learning approach to rare trajectory sampling](https://mlsanity.com/paper/2005.12890/) (2020)
- [Validation of Image-Based Neural Network Controllers through Adaptive Stress Testing](https://mlsanity.com/paper/2003.02381/) (2020)
- [Layered Adaptive Importance Sampling](https://mlsanity.com/paper/1505.04732/) (2015)
- [Falsification of Cyber-Physical Systems Using Deep Reinforcement Learning](https://mlsanity.com/paper/1805.00200/) (2018)
- [Scalable Autonomous Vehicle Safety Validation through Dynamic Programming and Scene Decomposition](https://mlsanity.com/paper/2004.06801/) (2020)

## Built on by (most cited first, 2 of 2)

- [Failure Probability Estimation for Black-Box Autonomous Systems using State-Dependent Importance Sampling Proposals](https://mlsanity.com/paper/2412.02154/) (2024)
- [Reinforcement Learning for Syndrome Extraction](https://mlsanity.com/paper/2609.12020/) (2026)

## Abstract (arXiv)

An important step in the design of autonomous systems is to evaluate the probability that a failure will occur. In safety-critical domains, the failure probability is extremely small so that the evaluation of a policy through Monte Carlo sampling is inefficient. Adaptive importance sampling approaches have been developed for rare event estimation but do not scale well to sequential systems with long horizons. In this work, we develop two adaptive importance sampling algorithms that can efficiently estimate the probability of rare events for sequential decision making systems. The basis for these algorithms is the minimization of the Kullback-Leibler divergence between a state-dependent proposal distribution and a target distribution over trajectories, but the resulting algorithms resemble policy gradient and value-based reinforcement learning. We apply multiple importance sampling to reduce the variance of our estimate and to address the issue of multi-modality in the optimal proposal distribution. We demonstrate our approach on a control task with both continuous and discrete actions spaces and show accuracy improvements over several baselines.