# Model-based Exploration of the Frontier of Behaviours for Deep Learning System Testing

arXiv: https://arxiv.org/abs/2007.02787 · HTML: https://mlsanity.com/paper/2007.02787/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jul 2020.

## Built on by (most cited first, 30 of 40)

- [Mind the Gap! A Study on the Transferability of Virtual vs Physical-world Testing of Autonomous Driving Systems](https://mlsanity.com/paper/2112.11255/) (2021)
- [DeepHyperion: Exploring the Feature Space of Deep Learning-Based Systems through Illumination Search](https://mlsanity.com/paper/2107.06997/) (2021)
- [Latent Imitator: Generating Natural Individual Discriminatory Instances for Black-Box Fairness Testing](https://mlsanity.com/paper/2305.11602/) (2023)
- [Many-Objective Reinforcement Learning for Online Testing of DNN-Enabled Systems](https://mlsanity.com/paper/2210.15432/) (2022)
- [When and Why Test Generators for Deep Learning Produce Invalid Inputs: an Empirical Study](https://mlsanity.com/paper/2212.11368/) (2022)
- [Benchmarking Safety Monitors for Image Classifiers with Machine Learning](https://mlsanity.com/paper/2110.01232/) (2021)
- [Testing of Deep Reinforcement Learning Agents with Surrogate Models](https://mlsanity.com/paper/2305.12751/) (2023)
- [Towards Practical Robustness Analysis for DNNs based on PAC-Model Learning](https://mlsanity.com/paper/2101.10102/) (2021)
- [DeepMetis: Augmenting a Deep Learning Test Set to Increase its Mutation Score](https://mlsanity.com/paper/2109.07514/) (2021)
- [Predicting Safety Misbehaviours in Autonomous Driving Systems using Uncertainty Quantification](https://mlsanity.com/paper/2404.18573/) (2024)
- [Efficient Domain Augmentation for Autonomous Driving Testing Using Diffusion Models](https://mlsanity.com/paper/2409.13661/) (2024)
- [Two is Better Than One: Digital Siblings to Improve Autonomous Driving Testing](https://mlsanity.com/paper/2305.08060/) (2023)
- [Hierarchical Distribution-Aware Testing of Deep Learning](https://mlsanity.com/paper/2205.08589/) (2022)
- [Generating and Detecting True Ambiguity: A Forgotten Danger in DNN Supervision Testing](https://mlsanity.com/paper/2207.10495/) (2022)
- [Requirements Engineering for Machine Learning: A Review and Reflection](https://mlsanity.com/paper/2210.00859/) (2022)
- [Boundary State Generation for Testing and Improvement of Autonomous Driving Systems](https://mlsanity.com/paper/2307.10590/) (2023)
- [Machine Learning Testing in an ADAS Case Study Using Simulation-Integrated Bio-Inspired Search-Based Testing](https://mlsanity.com/paper/2203.12026/) (2022)
- [Simulator-based explanation and debugging of hazard-triggering events in DNN-based safety-critical systems](https://mlsanity.com/paper/2204.00480/) (2022)
- [Bridging the Gap between Real-world and Synthetic Images for Testing Autonomous Driving Systems](https://mlsanity.com/paper/2408.13950/) (2024)
- [Benchmarking Generative AI Models for Deep Learning Test Input Generation](https://mlsanity.com/paper/2412.17652/) (2024)
- [Prioritizing Corners in OoD Detectors via Symbolic String Manipulation](https://mlsanity.com/paper/2205.07736/) (2022)
- [Targeted Deep Learning System Boundary Testing](https://mlsanity.com/paper/2408.06258/) (2024)
- [Supporting Safety Analysis of Image-processing DNNs through Clustering-based Approaches](https://mlsanity.com/paper/2301.13506/) (2023)
- [Identifying the Hazard Boundary of ML-enabled Autonomous Systems Using Cooperative Co-Evolutionary Search](https://mlsanity.com/paper/2301.13807/) (2023)
- [Reinforcement Learning for Online Testing of Autonomous Driving Systems: a Replication and Extension Study](https://mlsanity.com/paper/2403.13729/) (2024)
- [Finding Deviated Behaviors of the Compressed DNN Models for Image Classifications](https://mlsanity.com/paper/2112.02819/) (2021)
- [GIST: Generated Inputs Sets Transferability in Deep Learning](https://mlsanity.com/paper/2311.00801/) (2023)
- [Can Search-Based Testing with Pareto Optimization Effectively Cover Failure-Revealing Test Inputs?](https://mlsanity.com/paper/2410.11769/) (2024)
- [ARQ: A Mixed-Precision Quantization Framework for Accurate and Certifiably Robust DNNs](https://mlsanity.com/paper/2410.24214/) (2024)
- [Generating Critical Scenarios for Testing Automated Driving Systems](https://mlsanity.com/paper/2412.02574/) (2024)

## Abstract (arXiv)

With the increasing adoption of Deep Learning (DL) for critical tasks, such as autonomous driving, the evaluation of the quality of systems that rely on DL has become crucial. Once trained, DL systems produce an output for any arbitrary numeric vector provided as input, regardless of whether it is within or outside the validity domain of the system under test. Hence, the quality of such systems is determined by the intersection between their validity domain and the regions where their outputs exhibit a misbehaviour. In this paper, we introduce the notion of frontier of behaviours, i.e., the inputs at which the DL system starts to misbehave. If the frontier of misbehaviours is outside the validity domain of the system, the quality check is passed. Otherwise, the inputs at the intersection represent quality deficiencies of the system. We developed DeepJanus, a search-based tool that generates frontier inputs for DL systems. The experimental results obtained for the lane keeping component of a self-driving car show that the frontier of a well trained system contains almost exclusively unrealistic roads that violate the best practices of civil engineering, while the frontier of a poorly trained one includes many valid inputs that point to serious deficiencies of the system.