# Rare event failure test case generation in Learning-Enabled-Controllers

arXiv: https://arxiv.org/abs/2206.05533 · HTML: https://mlsanity.com/paper/2206.05533/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 21 references that are in the map.
- First posted on arXiv 11 Jun 2022.

## Builds on (most cited first, 9 of 9)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [Adversarial Attacks on Neural Network Policies](https://mlsanity.com/paper/1702.02284/) (2017)
- [Tactics of Adversarial Attack on Deep Reinforcement Learning Agents](https://mlsanity.com/paper/1703.06748/) (2017)
- [DeepBillboard: Systematic Physical-World Testing of Autonomous Driving Systems](https://mlsanity.com/paper/1812.10812/) (2018)

## Built on by (most cited first, 13 of 13)

- [LLM-Assisted Light: Leveraging Large Language Model Capabilities for Human-Mimetic Traffic Signal Control in Complex Urban Environments](https://mlsanity.com/paper/2403.08337/) (2024)
- [Data efficient surrogate modeling for engineering design: Ensemble-free batch mode deep active learning for regression](https://mlsanity.com/paper/2211.10360/) (2022)
- [Constrained Bayesian Optimization for Automatic Underwater Vehicle Hull Design](https://mlsanity.com/paper/2302.14732/) (2023)
- [Search for universal minimum drag resistance underwater vehicle hull using CFD](https://mlsanity.com/paper/2302.09441/) (2023)
- [Reduced Robust Random Cut Forest for Out-Of-Distribution detection in machine learning models](https://mlsanity.com/paper/2206.09247/) (2022)
- [Fusion of ML with numerical simulation for optimized propeller design](https://mlsanity.com/paper/2302.14740/) (2023)
- [Vehicle lateral control using Machine Learning for automated vehicle guidance](https://mlsanity.com/paper/2303.08187/) (2023)
- [Automatic AI controller that can drive with confidence: steering vehicle with uncertainty knowledge](https://mlsanity.com/paper/2404.16893/) (2024)
- [Navigating the Edge with the State-of-the-Art Insights into Corner Case Identification and Generation for Enhanced Autonomous Vehicle Safety](https://mlsanity.com/paper/2503.00077/) (2025)
- [Deep Active Learning for Regression Using ε-weighted Hybrid Query Strategy](https://mlsanity.com/paper/2206.13298/) (2022)
- [Systematic design space exploration by learning the explored space using Machine Learning](https://mlsanity.com/paper/2303.08249/) (2023)
- [Optimization for truss design using Bayesian optimization](https://mlsanity.com/paper/2306.01763/) (2023)
- [CORTEX-AVD: A Framework for CORner Case Testing and EXploration in Autonomous Vehicle Development](https://mlsanity.com/paper/2504.03989/) (2025)

## Abstract (arXiv)

Machine learning models have prevalent applications in many real-world problems, which increases the importance of correctness in the behaviour of these trained models. Finding a good test case that can reveal the potential failure in these trained systems can help to retrain these models to increase their correctness. For a well-trained model, the occurrence of a failure is rare. Consequently, searching these rare scenarios by evaluating each sample in input search space or randomized search would be costly and sometimes intractable due to large search space, limited computational resources, and available time. In this paper, we tried to address this challenge of finding these failure scenarios faster than traditional randomized search. The central idea of our approach is to separate the input data space in region of high failure probability and region of low/minimal failure probability based on the observation made by training data, data drawn from real-world statistics, and knowledge from a domain expert. Using these information, we can design a generative model from which we can generate scenarios that have a high likelihood to reveal the potential failure. We evaluated this approach on two different experimental scenarios and able to speed up the discovery of such failures a thousand-fold faster than the traditional randomized search.