mlsanityData to 14 Oct 2026

A Survey on Scenario-Based Testing for Automated Driving Systems in High-Fidelity Simulation

arXiv 2112.00964 ↗First posted 2 Dec 2021cs.SE cs.AI cs.LGcited by 21

Not on the map. Why?

Abstract (arXiv)

Automated Driving Systems (ADSs) have seen rapid progress in recent years. To ensure the safety and reliability of these systems, extensive testings are being conducted before their future mass deployment. Testing the system on the road is the closest to real-world and desirable approach, but it is incredibly costly. Also, it is infeasible to cover rare corner cases using such real-world testing. Thus, a popular alternative is to evaluate an ADS's performance in some well-designed challenging scenarios, a.k.a. scenario-based testing. High-fidelity simulators have been widely used in this setting to maximize flexibility and convenience in testing what-if scenarios. Although many works have been proposed offering diverse frameworks/methods for testing specific systems, the comparisons and connections among these works are still missing. To bridge this gap, in this work, we provide a generic formulation of scenario-based testing in high-fidelity simulation and conduct a literature review on the existing works. We further compare them and present the open challenges as well as potential future research directions.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Builds on 52 of its 140 references that are in the map. First posted on arXiv 2 Dec 2021.

Builds on

52 of 140 references.

References · 52

  1. 2015Continuous control with deep reinforcement learningcited by 3,407
  2. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  3. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  4. 2016Density estimation using Real NVPcited by 2,042
  5. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  6. 2016End to End Learning for Self-Driving Carscited by 1,009
  7. 2016Pixel Recurrent Neural Networkscited by 816
  8. 2017AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehiclescited by 523
  9. 2019Learning by Cheatingcited by 383
  10. 2017DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Carscited by 251
  11. 2015DeepDriving: Learning Affordance for Direct Perception in Autonomous Drivingcited by 231
  12. 2018Formal Security Analysis of Neural Networks using Symbolic Intervalscited by 185
38 more
  1. 2018Efficient Formal Safety Analysis of Neural Networkscited by 165
  2. 2019Adversarial Sensor Attack on LiDAR-based Perception in Autonomous Drivingcited by 152
  3. 2021AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehiclescited by 137
  4. 2017On a Formal Model of Safe and Scalable Self-driving Carscited by 131
  5. 2020LGSVL Simulator: A High Fidelity Simulator for Autonomous Drivingcited by 121
  6. 2018Scalable End-to-End Autonomous Vehicle Testing via Rare-event Simulationcited by 77
  7. 2020Learning to Collide: An Adaptive Safety-Critical Scenarios Generating Methodcited by 75
  8. 2020A Survey of Algorithms for Black-Box Safety Validation of Cyber-Physical Systemscited by 71
  9. 2020Dirty Road Can Attack: Security of Deep Learning based Automated Lane Centering under Physical-World Attackcited by 64
  10. 2020Multimodal Safety-Critical Scenarios Generation for Decision-Making Algorithms Evaluationcited by 55
  11. 2018DeepBillboard: Systematic Physical-World Testing of Autonomous Driving Systemscited by 50
  12. 2018Scenarios for Development, Test and Validation of Automated Vehiclescited by 48
  13. 2016Accelerated Evaluation of Automated Vehicles Safety in Lane Change Scenarios Based on Importance Sampling Techniquescited by 45
  14. 2021Exploring Adversarial Robustness of Multi-Sensor Perception Systems in Self Drivingcited by 45
  15. 2020Adversarial Evaluation of Autonomous Vehicles in Lane-Change Scenarioscited by 43
  16. 2017Ontology based Scene Creation for the Development of Automated Vehiclescited by 40
  17. 2019Failure-Scenario Maker for Rule-Based Agent using Multi-agent Adversarial Reinforcement Learning and its Application to Autonomous Drivingcited by 36
  18. 2020Neural Bridge Sampling for Evaluating Safety-Critical Autonomous Systemscited by 35
  19. 2021A Survey on Simulators for Testing Self-Driving Carscited by 27
  20. 2020Drift with Devil: Security of Multi-Sensor Fusion based Localization in High-Level Autonomous Driving under GPS Spoofing (Extended Version)cited by 25
  21. 2020Training Adversarial Agents to Exploit Weaknesses in Deep Control Policiescited by 24
  22. 2019Efficient Black-box Assessment of Autonomous Vehicle Safetycited by 23
  23. 2021Generating and Characterizing Scenarios for Safety Testing of Autonomous Vehiclescited by 21
  24. 2021Neural Network Guided Evolutionary Fuzzing for Finding Traffic Violations of Autonomous Vehiclescited by 19
  25. 2019Requirements-driven Test Generation for Autonomous Vehicles with Machine Learning Componentscited by 18
  26. 2019ML-based Fault Injection for Autonomous Vehicles: A Case for Bayesian Fault Injectioncited by 16
  27. 2020Fundamental Considerations around Scenario-Based Testing for Automated Drivingcited by 15
  28. 2021Finding Critical Scenarios for Automated Driving Systems: A Systematic Literature Reviewcited by 12
  29. 2017Accelerated Evaluation of Automated Vehicles Using Piecewise Mixture Modelscited by 8
  30. 2021Coverage based testing for V&V and Safety Assurance of Self-driving Autonomous Vehicles: A Systematic Literature Reviewcited by 8
  31. 2018Experimental Resilience Assessment of An Open-Source Driving Agentcited by 6
  32. 2019Metamorphic Testing for Object Detection Systemscited by 6
  33. 2017A Versatile Approach to Evaluating and Testing Automated Vehicles based on Kernel Methodscited by 5
  34. 2017Sequential Experimentation to Efficiently Test Automated Vehiclescited by 4
  35. 2019AVFI: Fault Injection for Autonomous Vehiclescited by 4
  36. 2021Coverage-based Scene Fuzzing for Virtual Autonomous Driving Testingcited by 4
  37. 2016Evaluation of Automated Vehicles in the Frontal Cut-in Scenario - an Enhanced Approach using Piecewise Mixture Modelscited by 2
  38. 2017An Accelerated Testing Approach for Automated Vehicles with Background Traffic Described by Joint Distributionscited by 2

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 5; 2025: 5; 2026: 6.

21 papers.

Sort:

By area:

  1. 2023CAT: Closed-loop Adversarial Training for Safe End-to-End Drivingcited by 54
  2. 2023Rethinking Closed-loop Training for Autonomous Drivingcited by 29
  3. 2024Efficient Domain Augmentation for Autonomous Driving Testing Using Diffusion Modelscited by 10
  4. 2023Domain Knowledge Distillation from Large Language Model: An Empirical Study in the Autonomous Driving Domaincited by 8
  5. 2025Trends in Motion Prediction Toward Deployable and Generalizable Autonomy: A Revisit and Perspectivescited by 5
  6. 2023Continual Driving Policy Optimization with Closed-Loop Individualized Curriculacited by 4
  7. 2024ROCAS: Root Cause Analysis of Autonomous Driving Accidents via Cyber-Physical Co-mutationcited by 3
  8. 2024Rate-Informed Discovery via Bayesian Adaptive Multifidelity Samplingcited by 3
  9. 2024Instance-Level Safety-Aware Fidelity of Synthetic Data and Its Calibrationcited by 2
  10. 2024Dance of the ADS: Orchestrating Failures through Historically-Informed Scenario Fuzzingcited by 2
  11. 2025Navigating the Edge with the State-of-the-Art Insights into Corner Case Identification and Generation for Enhanced Autonomous Vehicle Safetycited by 1
  12. 2023Semantic and Articulated Pedestrian Sensing Onboard a Moving Vehiclecited by 0
9 more
  1. 2025Towards provable probabilistic safety for scalable embodied AI systemscited by 0
  2. 2025Bayesian Optimization applied for accelerated Virtual Validation of the Autonomous Driving Functioncited by 0
  3. 2025SCOUT: A Lightweight Framework for Scenario Coverage Assessment in Autonomous Drivingcited by 0
  4. 2026Declarative Scenario-based Testing with RoadLogiccited by 0
  5. 2026Emergency Lane-Change Simulation: A Behavioral Guidance Approach for Risky Scenario Generationcited by 0
  6. 2026From Nominal Intensity to Equivalent Rainfall: A Path-Based Credibility Evaluation Framework for Simulated Rainfall in Autonomous-Driving Perception Testscited by 0
  7. 2026Revealing Safety-Critical Scenarios for UTM via Transformercited by 0
  8. 2026CUBICS: Situation-aware performance estimation for safety-relevant ML componentscited by 0
  9. 2026Evaluating Physical Consistency and Plausibility in Generative Scenario Models for Autonomous Drivingcited by 0