# Accelerated Evaluation of Automated Vehicles Safety in Lane Change Scenarios Based on Importance Sampling Techniques

arXiv: https://arxiv.org/abs/1605.04965 · HTML: https://mlsanity.com/paper/1605.04965/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 May 2016.

## Built on by (most cited first, 30 of 45)

- [SafeBench: A Benchmarking Platform for Safety Evaluation of Autonomous Vehicles](https://mlsanity.com/paper/2206.09682/) (2022)
- [Corner Case Generation and Analysis for Safety Assessment of Autonomous Vehicles](https://mlsanity.com/paper/2102.03483/) (2021)
- [Milestones in Autonomous Driving and Intelligent Vehicles Part I: Control, Computing System Design, Communication, HD Map, Testing, and Human Behaviors](https://mlsanity.com/paper/2305.11239/) (2023)
- [Generating and Characterizing Scenarios for Safety Testing of Autonomous Vehicles](https://mlsanity.com/paper/2103.07403/) (2021)
- [A Survey on Scenario-Based Testing for Automated Driving Systems in High-Fidelity Simulation](https://mlsanity.com/paper/2112.00964/) (2021)
- [CausalAF: Causal Autoregressive Flow for Safety-Critical Driving Scenario Generation](https://mlsanity.com/paper/2110.13939/) (2021)
- [Enhancing Autonomous Vehicle Training with Language Model Integration and Critical Scenario Generation](https://mlsanity.com/paper/2404.08570/) (2024)
- [LLM-attacker: Enhancing Closed-loop Adversarial Scenario Generation for Autonomous Driving with Large Language Models](https://mlsanity.com/paper/2501.15850/) (2025)
- [Finding Critical Scenarios for Automated Driving Systems: A Systematic Literature Review](https://mlsanity.com/paper/2110.08664/) (2021)
- [Safety of the Intended Driving Behavior Using Rulebooks](https://mlsanity.com/paper/2105.04472/) (2021)
- [Testing Rare Downstream Safety Violations via Upstream Adaptive Sampling of Perception Error Models](https://mlsanity.com/paper/2209.09674/) (2022)
- [Achieving Efficiency in Black Box Simulation of Distribution Tails with Self-structuring Importance Samplers](https://mlsanity.com/paper/2102.07060/) (2021)
- [Continual Driving Policy Optimization with Closed-Loop Individualized Curricula](https://mlsanity.com/paper/2309.14209/) (2023)
- [Make Full Use of Testing Information: An Integrated Accelerated Testing and Evaluation Method for Autonomous Driving Systems](https://mlsanity.com/paper/2501.11924/) (2025)
- [On the Need for a Statistical Foundation in Scenario-Based Testing of Autonomous Vehicles](https://mlsanity.com/paper/2505.02274/) (2025)
- [CounterScene: Counterfactual Causal Reasoning in Generative World Models for Safety-Critical Closed-Loop Evaluation](https://mlsanity.com/paper/2603.21104/) (2026)
- [I Know You Can't See Me: Dynamic Occlusion-Aware Safety Validation of Strategic Planners for Autonomous Vehicles Using Hypergames](https://mlsanity.com/paper/2109.09807/) (2021)
- [Certifiable Deep Importance Sampling for Rare-Event Simulation of Black-Box Systems](https://mlsanity.com/paper/2111.02204/) (2021)
- [Stackelberg Driver Model for Continual Policy Improvement in Scenario-Based Closed-Loop Autonomous Driving](https://mlsanity.com/paper/2309.14235/) (2023)
- [Importance Sampling-Guided Meta-Training for Intelligent Agents in Highly Interactive Environments](https://mlsanity.com/paper/2407.15839/) (2024)
- [Scalable Safety-Critical Policy Evaluation with Accelerated Rare Event Sampling](https://mlsanity.com/paper/2106.10566/) (2021)
- [Few-Shot Scenario Testing for Autonomous Vehicles Based on Neighborhood Coverage and Similarity](https://mlsanity.com/paper/2402.01795/) (2024)
- [Adaptive Testing Environment Generation for Connected and Automated Vehicles with Dense Reinforcement Learning](https://mlsanity.com/paper/2402.19275/) (2024)
- [Exploring Critical Testing Scenarios for Decision-Making Policies: An LLM Approach](https://mlsanity.com/paper/2412.06684/) (2024)
- [Calibrating Over-Parametrized Simulation Models: A Framework via Eligibility Set](https://mlsanity.com/paper/2105.12893/) (2021)
- [Test Against High-Dimensional Uncertainties: Accelerated Evaluation of Autonomous Vehicles with Deep Importance Sampling](https://mlsanity.com/paper/2204.02351/) (2022)
- [STEAM & MoSAFE: SOTIF Error-and-Failure Model & Analysis for AI-Enabled Driving Automation](https://mlsanity.com/paper/2312.09559/) (2023)
- [Flow to Rare Events: An Application of Normalizing Flow in Temporal Importance Sampling for Automated Vehicle Validation](https://mlsanity.com/paper/2407.07320/) (2024)
- [Post-Convergence Sim-to-Real Policy Transfer: A Principled Alternative to Cherry-Picking](https://mlsanity.com/paper/2504.15414/) (2025)
- [Learning from Risk: LLM-Guided Generation of Safety-Critical Scenarios with Prior Knowledge](https://mlsanity.com/paper/2511.20726/) (2025)

## Abstract (arXiv)

Automated vehicles (AVs) must be evaluated thoroughly before their release and deployment. A widely-used evaluation approach is the Naturalistic-Field Operational Test (N-FOT), which tests prototype vehicles directly on the public roads. Due to the low exposure to safety-critical scenarios, N-FOTs are time-consuming and expensive to conduct. In this paper, we propose an accelerated evaluation approach for AVs. The results can be used to generate motions of the primary other vehicles to accelerate the verification of AVs in simulations and controlled experiments. Frontal collision due to unsafe cut-ins is the target crash type of this paper. Human-controlled vehicles making unsafe lane changes are modeled as the primary disturbance to AVs based on data collected by the University of Michigan Safety Pilot Model Deployment Program. The cut-in scenarios are generated based on skewed statistics of collected human driver behaviors, which generate risky testing scenarios while preserving the statistical information so that the safety benefits of AVs in non-accelerated cases can be accurately estimated. The Cross Entropy method is used to recursively search for the optimal skewing parameters. The frequencies of occurrence of conflicts, crashes and injuries are estimated for a modeled automated vehicle, and the achieved accelerated rate is around 2,000 to 20,000. In other words, in the accelerated simulations, driving for 1,000 miles will expose the AV with challenging scenarios that will take about 2 to 20 million miles of real-world driving to encounter. This technique thus has the potential to reduce greatly the development and validation time for AVs.