# Generating and Characterizing Scenarios for Safety Testing of Autonomous Vehicles

arXiv: https://arxiv.org/abs/2103.07403 · HTML: https://mlsanity.com/paper/2103.07403/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 18 references that are in the map.
- First posted on arXiv 12 Mar 2021.

## Builds on (most cited first, 9 of 9)

- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Scenic: A Language for Scenario Specification and Scene Generation](https://mlsanity.com/paper/1809.09310/) (2018)
- [Scalable End-to-End Autonomous Vehicle Testing via Rare-event Simulation](https://mlsanity.com/paper/1811.00145/) (2018)
- [Model-Predictive Policy Learning with Uncertainty Regularization for Driving in Dense Traffic](https://mlsanity.com/paper/1901.02705/) (2019)
- [Accelerated Evaluation of Automated Vehicles Safety in Lane Change Scenarios Based on Importance Sampling Techniques](https://mlsanity.com/paper/1605.04965/) (2016)
- [Adaptive Stress Testing with Reward Augmentation for Autonomous Vehicle Validation](https://mlsanity.com/paper/1908.01046/) (2019)
- [Compositional Falsification of Cyber-Physical Systems with Machine Learning Components](https://mlsanity.com/paper/1703.00978/) (2017)
- [Using control synthesis to generate corner cases: A case study on autonomous driving](https://mlsanity.com/paper/1807.09537/) (2018)
- [AVFI: Fault Injection for Autonomous Vehicles](https://mlsanity.com/paper/1907.01038/) (2019)

## Built on by (most cited first, 21 of 21)

- [Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic Prior](https://mlsanity.com/paper/2112.05077/) (2021)
- [EgoGen: An Egocentric Synthetic Data Generator](https://mlsanity.com/paper/2401.08739/) (2024)
- [Choose Your Simulator Wisely: A Review on Open-source Simulators for Autonomous Driving](https://mlsanity.com/paper/2311.11056/) (2023)
- [A Survey on Scenario-Based Testing for Automated Driving Systems in High-Fidelity Simulation](https://mlsanity.com/paper/2112.00964/) (2021)
- [Selective Prediction via Training Dynamics](https://mlsanity.com/paper/2205.13532/) (2022)
- [Text2Scenario: Text-Driven Scenario Generation for Autonomous Driving Test](https://mlsanity.com/paper/2503.02911/) (2025)
- [Safety-Critical Scenario Generation Via Reinforcement Learning Based Editing](https://mlsanity.com/paper/2306.14131/) (2023)
- [Learning to Drive via Asymmetric Self-Play](https://mlsanity.com/paper/2409.18218/) (2024)
- [Confidence-aware Contrastive Learning for Selective Classification](https://mlsanity.com/paper/2406.04745/) (2024)
- [Training Private Models That Know What They Don't Know](https://mlsanity.com/paper/2305.18393/) (2023)
- [INSIGHT: Enhancing Autonomous Driving Safety through Vision-Language Models on Context-Aware Hazard Detection and Edge Case Evaluation](https://mlsanity.com/paper/2502.00262/) (2025)
- [Causal Scene BERT: Improving object detection by searching for challenging groups of data](https://mlsanity.com/paper/2202.03651/) (2022)
- [p-DkNN: Out-of-Distribution Detection Through Statistical Testing of Deep Representations](https://mlsanity.com/paper/2207.12545/) (2022)
- [From Failures to Fixes: LLM-Driven Scenario Repair for Self-Evolving Autonomous Driving](https://mlsanity.com/paper/2505.22067/) (2025)
- [Confidential Guardian: Cryptographically Prohibiting the Abuse of Model Abstention](https://mlsanity.com/paper/2505.23968/) (2025)
- [What Does It Take to Build a Performant Selective Classifier?](https://mlsanity.com/paper/2510.20242/) (2025)
- [Navigating the Edge with the State-of-the-Art Insights into Corner Case Identification and Generation for Enhanced Autonomous Vehicle Safety](https://mlsanity.com/paper/2503.00077/) (2025)
- [CARJAN: Agent-Based Generation and Simulation of Traffic Scenarios with AJAN](https://mlsanity.com/paper/2508.21411/) (2025)
- [ScenePilot: Controllable Boundary-Driven Critical Scenario Generation for Autonomous Driving](https://mlsanity.com/paper/2605.21168/) (2026)
- [From Driving Videos to Simulatable Scenarios](https://mlsanity.com/paper/2606.21993/) (2026)
- [Collision Snapshot Guided Time-Reversed Safety-Critical Scenario Generation](https://mlsanity.com/paper/2609.06433/) (2026)

## Abstract (arXiv)

Extracting interesting scenarios from real-world data as well as generating failure cases is important for the development and testing of autonomous systems. We propose efficient mechanisms to both characterize and generate testing scenarios using a state-of-the-art driving simulator. For any scenario, our method generates a set of possible driving paths and identifies all the possible safe driving trajectories that can be taken starting at different times, to compute metrics that quantify the complexity of the scenario. We use our method to characterize real driving data from the Next Generation Simulation (NGSIM) project, as well as adversarial scenarios generated in simulation. We rank the scenarios by defining metrics based on the complexity of avoiding accidents and provide insights into how the AV could have minimized the probability of incurring an accident. We demonstrate a strong correlation between the proposed metrics and human intuition.