# Finding Failures in High-Fidelity Simulation using Adaptive Stress Testing and the Backward Algorithm

arXiv: https://arxiv.org/abs/2107.12940 · HTML: https://mlsanity.com/paper/2107.12940/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 21 references that are in the map.
- First posted on arXiv 27 Jul 2021.

## Builds on (most cited first, 13 of 13)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Learning Montezuma's Revenge from a Single Demonstration](https://mlsanity.com/paper/1812.03381/) (2018)
- [Adaptive Stress Testing for Autonomous Vehicles](https://mlsanity.com/paper/1902.01909/) (2019)
- [A Survey of Algorithms for Black-Box Safety Validation of Cyber-Physical Systems](https://mlsanity.com/paper/2005.02979/) (2020)
- [Dynamic Occupancy Grid Prediction for Urban Autonomous Driving: A Deep Learning Approach with Fully Automatic Labeling](https://mlsanity.com/paper/1705.08781/) (2017)
- [Dynamic Environment Prediction in Urban Scenes using Recurrent Representation Learning](https://mlsanity.com/paper/1904.12374/) (2019)
- [A Random Finite Set Approach for Dynamic Occupancy Grid Maps with Real-Time Application](https://mlsanity.com/paper/1605.02406/) (2016)
- [Falsification of Cyber-Physical Systems Using Deep Reinforcement Learning](https://mlsanity.com/paper/1805.00200/) (2018)
- [Efficient Autonomy Validation in Simulation with Adaptive Stress Testing](https://mlsanity.com/paper/1907.06795/) (2019)
- [Adaptive Stress Testing without Domain Heuristics using Go-Explore](https://mlsanity.com/paper/2004.04292/) (2020)
- [The Adaptive Stress Testing Formulation](https://mlsanity.com/paper/2004.04293/) (2020)

## Built on by (most cited first, 7 of 7)

- [Falsification of Learning-Based Controllers through Multi-Fidelity Bayesian Optimization](https://mlsanity.com/paper/2212.14118/) (2022)
- [AuthSim: Towards Authentic and Effective Safety-critical Scenario Generation for Autonomous Driving Tests](https://mlsanity.com/paper/2502.21100/) (2025)
- [Driving in Corner Case: A Real-World Adversarial Closed-Loop Evaluation Platform for End-to-End Autonomous Driving](https://mlsanity.com/paper/2512.16055/) (2025)
- [A Real-time Critical-scenario-generation Framework for Testing Autonomous Driving System](https://mlsanity.com/paper/2206.00910/) (2022)
- [Vectorized Scenario Description and Motion Prediction for Scenario-Based Testing](https://mlsanity.com/paper/2302.01161/) (2023)
- [ReMAV: Reward Modeling of Autonomous Vehicles for Finding Likely Failure Events](https://mlsanity.com/paper/2308.14550/) (2023)
- [Scenario Generation for Testing of Autonomous Driving Systems Using Real-World Failure Records](https://mlsanity.com/paper/2606.31131/) (2026)

## Abstract (arXiv)

Validating the safety of autonomous systems generally requires the use of high-fidelity simulators that adequately capture the variability of real-world scenarios. However, it is generally not feasible to exhaustively search the space of simulation scenarios for failures. Adaptive stress testing (AST) is a method that uses reinforcement learning to find the most likely failure of a system. AST with a deep reinforcement learning solver has been shown to be effective in finding failures across a range of different systems. This approach generally involves running many simulations, which can be very expensive when using a high-fidelity simulator. To improve efficiency, we present a method that first finds failures in a low-fidelity simulator. It then uses the backward algorithm, which trains a deep neural network policy using a single expert demonstration, to adapt the low-fidelity failures to high-fidelity. We have created a series of autonomous vehicle validation case studies that represent some of the ways low-fidelity and high-fidelity simulators can differ, such as time discretization. We demonstrate in a variety of case studies that this new AST approach is able to find failures with significantly fewer high-fidelity simulation steps than are needed when just running AST directly in high-fidelity. As a proof of concept, we also demonstrate AST on NVIDIA's DriveSim simulator, an industry state-of-the-art high-fidelity simulator for finding failures in autonomous vehicles.