# DeepEvolution: A Search-Based Testing Approach for Deep Neural Networks

arXiv: https://arxiv.org/abs/1909.02563 · HTML: https://mlsanity.com/paper/1909.02563/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Sep 2019.

## Built on by (most cited first, 15 of 15)

- [How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review](https://mlsanity.com/paper/2107.12045/) (2021)
- [Automatic Fault Detection for Deep Learning Programs Using Graph Transformations](https://mlsanity.com/paper/2105.08095/) (2021)
- [Aries: Efficient Testing of Deep Neural Networks via Labeling-Free Accuracy Estimation](https://mlsanity.com/paper/2207.10942/) (2022)
- [Machine Learning Robustness: A Primer](https://mlsanity.com/paper/2404.00897/) (2024)
- [FedDebug: Systematic Debugging for Federated Learning Applications](https://mlsanity.com/paper/2301.03553/) (2023)
- [Benchmarking Generative AI Models for Deep Learning Test Input Generation](https://mlsanity.com/paper/2412.17652/) (2024)
- [DiverGet: A Search-Based Software Testing Approach for Deep Neural Network Quantization Assessment](https://mlsanity.com/paper/2207.06282/) (2022)
- [Testing Feedforward Neural Networks Training Programs](https://mlsanity.com/paper/2204.00694/) (2022)
- [Physics-Guided Adversarial Machine Learning for Aircraft Systems Simulation](https://mlsanity.com/paper/2209.03431/) (2022)
- [GIST: Generated Inputs Sets Transferability in Deep Learning](https://mlsanity.com/paper/2311.00801/) (2023)
- [HOMRS: High Order Metamorphic Relations Selector for Deep Neural Networks](https://mlsanity.com/paper/2107.04863/) (2021)
- [Models of Computational Profiles to Study the Likelihood of DNN Metamorphic Test Cases](https://mlsanity.com/paper/2107.13491/) (2021)
- [Trimming the Risk: Towards Reliable Continuous Training for Deep Learning Inspection Systems](https://mlsanity.com/paper/2409.09108/) (2024)
- [DiffGAN: A Test Generation Approach for Differential Testing of Deep Neural Networks for Image Analysis](https://mlsanity.com/paper/2410.19794/) (2024)
- [Towards Assessing Deep Learning Test Input Generators](https://mlsanity.com/paper/2504.02329/) (2025)

## Abstract (arXiv)

The increasing inclusion of Deep Learning (DL) models in safety-critical systems such as autonomous vehicles have led to the development of multiple model-based DL testing techniques. One common denominator of these testing techniques is the automated generation of test cases, e.g., new inputs transformed from the original training data with the aim to optimize some test adequacy criteria. So far, the effectiveness of these approaches has been hindered by their reliance on random fuzzing or transformations that do not always produce test cases with a good diversity. To overcome these limitations, we propose, DeepEvolution, a novel search-based approach for testing DL models that relies on metaheuristics to ensure a maximum diversity in generated test cases. We assess the effectiveness of DeepEvolution in testing computer-vision DL models and found that it significantly increases the neuronal coverage of generated test cases. Moreover, using DeepEvolution, we could successfully find several corner-case behaviors. Finally, DeepEvolution outperformed Tensorfuzz (a coverage-guided fuzzing tool developed at Google Brain) in detecting latent defects introduced during the quantization of the models. These results suggest that search-based approaches can help build effective testing tools for DL systems.