# Requirements-driven Test Generation for Autonomous Vehicles with Machine Learning Components

arXiv: https://arxiv.org/abs/1908.01094 · HTML: https://mlsanity.com/paper/1908.01094/

## Key facts

- Cited by 17 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 2 Aug 2019.

## Built on by (most cited first, 17 of 17)

- [Software Engineering for AI-Based Systems: A Survey](https://mlsanity.com/paper/2105.01984/) (2021)
- [Explainable AI for Safe and Trustworthy Autonomous Driving: A Systematic Review](https://mlsanity.com/paper/2402.10086/) (2024)
- [Description of Corner Cases in Automated Driving: Goals and Challenges](https://mlsanity.com/paper/2109.09607/) (2021)
- [A Survey on Scenario-Based Testing for Automated Driving Systems in High-Fidelity Simulation](https://mlsanity.com/paper/2112.00964/) (2021)
- [Finding Critical Scenarios for Automated Driving Systems: A Systematic Literature Review](https://mlsanity.com/paper/2110.08664/) (2021)
- [Requirements Engineering Framework for Human-centered Artificial Intelligence Software Systems](https://mlsanity.com/paper/2303.02920/) (2023)
- [From Dashcam Videos to Driving Simulations: Stress Testing Automated Vehicles against Rare Events](https://mlsanity.com/paper/2411.16027/) (2024)
- [Formalizing and Evaluating Requirements of Perception Systems for Automated Vehicles using Spatio-Temporal Perception Logic](https://mlsanity.com/paper/2206.14372/) (2022)
- [Local Repair of Neural Networks Using Optimization](https://mlsanity.com/paper/2109.14041/) (2021)
- [Adaptive Testing Environment Generation for Connected and Automated Vehicles with Dense Reinforcement Learning](https://mlsanity.com/paper/2402.19275/) (2024)
- [Rationale-aware Autonomous Driving Policy utilizing Safety Force Field implemented on CARLA Simulator](https://mlsanity.com/paper/2211.10237/) (2022)
- [How to Evaluate Entity Resolution Systems: An Entity-Centric Framework with Application to Inventor Name Disambiguation](https://mlsanity.com/paper/2404.05622/) (2024)
- [VCAT: Vulnerability-aware and Curiosity-driven Adversarial Training for Enhancing Autonomous Vehicle Robustness](https://mlsanity.com/paper/2409.12997/) (2024)
- [Navigating the Edge with the State-of-the-Art Insights into Corner Case Identification and Generation for Enhanced Autonomous Vehicle Safety](https://mlsanity.com/paper/2503.00077/) (2025)
- [Towards Counterfactual Explanation and Assertion Inference for CPS Debugging](https://mlsanity.com/paper/2604.07679/) (2026)
- [Efficient Safety Testing of Autonomous Vehicles via Adaptive Search over Crash-Derived Scenarios](https://mlsanity.com/paper/2508.06575/) (2025)
- [FeedbackLLM: Metadata driven Multi-Agentic Language Agnostic Test Case Generator with Evolving prompt and Coverage Feedback](https://mlsanity.com/paper/2605.01264/) (2026)

## Abstract (arXiv)

Autonomous vehicles are complex systems that are challenging to test and debug. A requirements-driven approach to the development process can decrease the resources required to design and test these systems, while simultaneously increasing the reliability. We present a testing framework that uses signal temporal logic (STL), which is a precise and unambiguous requirements language. Our framework evaluates test cases against the STL formulae and additionally uses the requirements to automatically identify test cases that fail to satisfy the requirements. One of the key features of our tool is the support for machine learning (ML) components in the system design, such as deep neural networks. The framework allows evaluation of the control algorithms, including the ML components, and it also includes models of CCD camera, lidar, and radar sensors, as well as the vehicle environment. We use multiple methods to generate test cases, including covering arrays, which is an efficient method to search discrete variable spaces. The resulting test cases can be used to debug the controller design by identifying controller behaviors that do not satisfy requirements. The test cases can also enhance the testing phase of development by identifying critical corner cases that correspond to the limits of the system's allowed behaviors. We present STL requirements for an autonomous vehicle system, which capture both component-level and system-level behaviors. Additionally, we present three driving scenarios and demonstrate how our requirements-driven testing framework can be used to identify critical system behaviors, which can be used to support the development process.