# Efficient Black-box Assessment of Autonomous Vehicle Safety

arXiv: https://arxiv.org/abs/1912.03618 · HTML: https://mlsanity.com/paper/1912.03618/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Dec 2019.

## Built on by (most cited first, 23 of 23)

- [AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehicles](https://mlsanity.com/paper/2101.06549/) (2021)
- [KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradients](https://mlsanity.com/paper/2204.13683/) (2022)
- [V2XP-ASG: Generating Adversarial Scenes for Vehicle-to-Everything Perception](https://mlsanity.com/paper/2209.13679/) (2022)
- [A Survey on Scenario-Based Testing for Automated Driving Systems in High-Fidelity Simulation](https://mlsanity.com/paper/2112.00964/) (2021)
- [Neural Network Guided Evolutionary Fuzzing for Finding Traffic Violations of Autonomous Vehicles](https://mlsanity.com/paper/2109.06126/) (2021)
- [Adversarial Safety-Critical Scenario Generation using Naturalistic Human Driving Priors](https://mlsanity.com/paper/2408.03200/) (2024)
- [Learning to Drive via Asymmetric Self-Play](https://mlsanity.com/paper/2409.18218/) (2024)
- [Adv3D: Generating Safety-Critical 3D Objects through Closed-Loop Simulation](https://mlsanity.com/paper/2311.01446/) (2023)
- [Detecting Multi-Sensor Fusion Errors in Advanced Driver-Assistance Systems](https://mlsanity.com/paper/2109.06404/) (2021)
- [Testing Rare Downstream Safety Violations via Upstream Adaptive Sampling of Perception Error Models](https://mlsanity.com/paper/2209.09674/) (2022)
- [Diffusion-Based Failure Sampling for Evaluating Safety-Critical Autonomous Systems](https://mlsanity.com/paper/2406.14761/) (2024)
- [Self-Improving Safety Performance of Reinforcement Learning Based Driving with Black-Box Verification Algorithms](https://mlsanity.com/paper/2210.16575/) (2022)
- [Model-based Validation as Probabilistic Inference](https://mlsanity.com/paper/2305.09930/) (2023)
- [Rate-Informed Discovery via Bayesian Adaptive Multifidelity Sampling](https://mlsanity.com/paper/2411.17826/) (2024)
- [Identifying the Hazard Boundary of ML-enabled Autonomous Systems Using Cooperative Co-Evolutionary Search](https://mlsanity.com/paper/2301.13807/) (2023)
- [Formal and Practical Elements for the Certification of Machine Learning Systems](https://mlsanity.com/paper/2310.03217/) (2023)
- [Adaptive Testing Environment Generation for Connected and Automated Vehicles with Dense Reinforcement Learning](https://mlsanity.com/paper/2402.19275/) (2024)
- [Fast nonlinear risk assessment for autonomous vehicles using learned conditional probabilistic models of agent futures](https://mlsanity.com/paper/2109.09975/) (2021)
- [Evaluating Automated Driving Planner Robustness against Adversarial Influence](https://mlsanity.com/paper/2205.14697/) (2022)
- [Exiting the Simulation: The Road to Robust and Resilient Autonomous Vehicles at Scale](https://mlsanity.com/paper/2210.10876/) (2022)
- [Failure Probability Estimation for Black-Box Autonomous Systems using State-Dependent Importance Sampling Proposals](https://mlsanity.com/paper/2412.02154/) (2024)
- [Scalable Importance Sampling in High Dimensions with Low-Rank Mixture Proposals](https://mlsanity.com/paper/2505.13335/) (2025)
- [Single-Eye View: Monocular Real-time Perception Package for Autonomous Driving](https://mlsanity.com/paper/2603.21061/) (2026)

## Abstract (arXiv)

While autonomous vehicle (AV) technology has shown substantial progress, we still lack tools for rigorous and scalable testing. Real-world testing, the textitde-facto evaluation method, is dangerous to the public. Moreover, due to the rare nature of failures, billions of miles of driving are needed to statistically validate performance claims. Thus, the industry has largely turned to simulation to evaluate AV systems. However, having a simulation stack alone is not a solution. A simulation testing framework needs to prioritize which scenarios to run, learn how the chosen scenarios provide coverage of failure modes, and rank failure scenarios in order of importance. We implement a simulation testing framework that evaluates an entire modern AV system as a black box. This framework estimates the probability of accidents under a base distribution governing standard traffic behavior. In order to accelerate rare-event probability evaluation, we efficiently learn to identify and rank failure scenarios via adaptive importance-sampling methods. Using this framework, we conduct the first independent evaluation of a full-stack commercial AV system, Comma AI's OpenPilot.