# Neural Network Guided Evolutionary Fuzzing for Finding Traffic Violations of Autonomous Vehicles

arXiv: https://arxiv.org/abs/2109.06126 · HTML: https://mlsanity.com/paper/2109.06126/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Placed in Adversarial scenarios and safety testing, one of 9 directions in End-to-end autonomous driving.
- Builds on 27 of its 99 references that are in the map.
- First posted on arXiv 13 Sep 2021.

## Placed in

- End-to-end autonomous driving / [Adversarial scenarios and safety testing](https://mlsanity.com/driving/adversarial-scenarios-and-safety-testing/)

## Builds on (most cited first, 27 of 27)

- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Learning by Cheating](https://mlsanity.com/paper/1912.12294/) (2019)
- [DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars](https://mlsanity.com/paper/1708.08559/) (2017)
- [pymoo: Multi-objective Optimization in Python](https://mlsanity.com/paper/2002.04504/) (2020)
- [Adversarial Sensor Attack on LiDAR-based Perception in Autonomous Driving](https://mlsanity.com/paper/1907.06826/) (2019)
- [AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehicles](https://mlsanity.com/paper/2101.06549/) (2021)
- [LGSVL Simulator: A High Fidelity Simulator for Autonomous Driving](https://mlsanity.com/paper/2005.03778/) (2020)
- [Scalable End-to-End Autonomous Vehicle Testing via Rare-event Simulation](https://mlsanity.com/paper/1811.00145/) (2018)
- [Learning to Collide: An Adaptive Safety-Critical Scenarios Generating Method](https://mlsanity.com/paper/2003.01197/) (2020)
- [Adaptive Stress Testing for Autonomous Vehicles](https://mlsanity.com/paper/1902.01909/) (2019)
- [Multimodal Safety-Critical Scenarios Generation for Decision-Making Algorithms Evaluation](https://mlsanity.com/paper/2009.08311/) (2020)
- [DeepBillboard: Systematic Physical-World Testing of Autonomous Driving Systems](https://mlsanity.com/paper/1812.10812/) (2018)
- [Scenarios for Development, Test and Validation of Automated Vehicles](https://mlsanity.com/paper/1801.08598/) (2018)
- [Exploring Adversarial Robustness of Multi-Sensor Perception Systems in Self Driving](https://mlsanity.com/paper/2101.06784/) (2021)
- [Adversarial Evaluation of Autonomous Vehicles in Lane-Change Scenarios](https://mlsanity.com/paper/2004.06531/) (2020)
- [Drift with Devil: Security of Multi-Sensor Fusion based Localization in High-Level Autonomous Driving under GPS Spoofing (Extended Version)](https://mlsanity.com/paper/2006.10318/) (2020)
- [Training Adversarial Agents to Exploit Weaknesses in Deep Control Policies](https://mlsanity.com/paper/2002.12078/) (2020)
- [Efficient Black-box Assessment of Autonomous Vehicle Safety](https://mlsanity.com/paper/1912.03618/) (2019)
- [Angora: Efficient Fuzzing by Principled Search](https://mlsanity.com/paper/1803.01307/) (2018)
- [ConAML: Constrained Adversarial Machine Learning for Cyber-Physical Systems](https://mlsanity.com/paper/2003.05631/) (2020)
- [Testing the Safety of Self-driving Vehicles by Simulating Perception and Prediction](https://mlsanity.com/paper/2008.06020/) (2020)
- [VERIFAI: A Toolkit for the Design and Analysis of Artificial Intelligence-Based Systems](https://mlsanity.com/paper/1902.04245/) (2019)
- [MTFuzz: Fuzzing with a Multi-Task Neural Network](https://mlsanity.com/paper/2005.12392/) (2020)
- [Pythia: Grammar-Based Fuzzing of REST APIs with Coverage-guided Feedback and Learning-based Mutations](https://mlsanity.com/paper/2005.11498/) (2020)
- [Evolutionary Grammar-Based Fuzzing](https://mlsanity.com/paper/2008.01150/) (2020)
- [The Relevance of Classic Fuzz Testing: Have We Solved This One?](https://mlsanity.com/paper/2008.06537/) (2020)

## Built on by (most cited first, 19 of 19)

- [A Survey on Scenario-Based Testing for Automated Driving Systems in High-Fidelity Simulation](https://mlsanity.com/paper/2112.00964/) (2021)
- [Two is Better Than One: Digital Siblings to Improve Autonomous Driving Testing](https://mlsanity.com/paper/2305.08060/) (2023)
- [Detecting Multi-Sensor Fusion Errors in Advanced Driver-Assistance Systems](https://mlsanity.com/paper/2109.06404/) (2021)
- [Testing learning-enabled cyber-physical systems with Large-Language Models: A Formal Approach](https://mlsanity.com/paper/2311.07377/) (2023)
- [Evaluating the Impact of Flaky Simulators on Testing Autonomous Driving Systems](https://mlsanity.com/paper/2311.18768/) (2023)
- [Decictor: Towards Evaluating the Robustness of Decision-Making in Autonomous Driving Systems](https://mlsanity.com/paper/2402.18393/) (2024)
- [Dance of the ADS: Orchestrating Failures through Historically-Informed Scenario Fuzzing](https://mlsanity.com/paper/2407.04359/) (2024)
- [Exploring Critical Testing Scenarios for Decision-Making Policies: An LLM Approach](https://mlsanity.com/paper/2412.06684/) (2024)
- [The Integration of Machine Learning into Automated Test Generation: A Systematic Mapping Study](https://mlsanity.com/paper/2206.10210/) (2022)
- [Navigating the Edge with the State-of-the-Art Insights into Corner Case Identification and Generation for Enhanced Autonomous Vehicle Safety](https://mlsanity.com/paper/2503.00077/) (2025)
- [Simulator Ensembles for Trustworthy Autonomous Driving Systems Testing](https://mlsanity.com/paper/2503.08936/) (2025)
- [ADReFT: Adaptive Decision Repair for Safe Autonomous Driving via Reinforcement Fine-Tuning](https://mlsanity.com/paper/2506.23960/) (2025)
- [Advancing Autonomous Driving System Testing: Demands, Challenges, and Future Directions](https://mlsanity.com/paper/2512.11887/) (2025)
- [ReMAV: Reward Modeling of Autonomous Vehicles for Finding Likely Failure Events](https://mlsanity.com/paper/2308.14550/) (2023)
- [Semantic and Articulated Pedestrian Sensing Onboard a Moving Vehicle](https://mlsanity.com/paper/2309.06313/) (2023)
- [Natural Adversaries: Fuzzing Autonomous Vehicles with Realistic Roadside Object Placements](https://mlsanity.com/paper/2409.10562/) (2024)
- [STCLocker: Deadlock Avoidance Testing for Autonomous Driving Systems](https://mlsanity.com/paper/2506.23995/) (2025)
- [When Autonomous Vehicle Meets V2X Cooperative Perception: How Far Are We?](https://mlsanity.com/paper/2509.24927/) (2025)
- [From Particles to Perils: SVGD-Based Hazardous Scenario Generation for Autonomous Driving Systems Testing](https://mlsanity.com/paper/2604.18918/) (2026)

## Abstract (arXiv)

Self-driving cars and trucks, autonomous vehicles (AVs), should not be accepted by regulatory bodies and the public until they have much higher confidence in their safety and reliability -- which can most practically and convincingly be achieved by testing. But existing testing methods are inadequate for checking the end-to-end behaviors of AV controllers against complex, real-world corner cases involving interactions with multiple independent agents such as pedestrians and human-driven vehicles. While test-driving AVs on streets and highways fails to capture many rare events, existing simulation-based testing methods mainly focus on simple scenarios and do not scale well for complex driving situations that require sophisticated awareness of the surroundings. To address these limitations, we propose a new fuzz testing technique, called AutoFuzz, which can leverage widely-used AV simulators' API grammars to generate semantically and temporally valid complex driving scenarios (sequences of scenes). To efficiently search for traffic violations-inducing scenarios in a large search space, we propose a constrained neural network (NN) evolutionary search method to optimize AutoFuzz. Evaluation of our prototype on one state-of-the-art learning-based controller, two rule-based controllers, and one industrial-grade controller in five scenarios shows that AutoFuzz efficiently finds hundreds of traffic violations in high-fidelity simulation environments. For each scenario, AutoFuzz can find on average 10-39% more unique traffic violations than the best-performing baseline method. Further, fine-tuning the learning-based controller with the traffic violations found by AutoFuzz successfully reduced the traffic violations found in the new version of the AV controller software.