# DeepHyperion: Exploring the Feature Space of Deep Learning-Based Systems through Illumination Search

arXiv: https://arxiv.org/abs/2107.06997 · HTML: https://mlsanity.com/paper/2107.06997/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 41 references that are in the map.
- First posted on arXiv 5 Jul 2021.

## Builds on (most cited first, 12 of 12)

- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [Illuminating search spaces by mapping elites](https://mlsanity.com/paper/1504.04909/) (2015)
- [DeepXplore: Automated Whitebox Testing of Deep Learning Systems](https://mlsanity.com/paper/1705.06640/) (2017)
- [DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars](https://mlsanity.com/paper/1708.08559/) (2017)
- [Data Programming: Creating Large Training Sets, Quickly](https://mlsanity.com/paper/1605.07723/) (2016)
- [Machine Learning Testing: Survey, Landscapes and Horizons](https://mlsanity.com/paper/1906.10742/) (2019)
- [DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems](https://mlsanity.com/paper/1803.07519/) (2018)
- [Guiding Deep Learning System Testing using Surprise Adequacy](https://mlsanity.com/paper/1808.08444/) (2018)
- [Taxonomy of Real Faults in Deep Learning Systems](https://mlsanity.com/paper/1910.11015/) (2019)
- [DLFuzz: Differential Fuzzing Testing of Deep Learning Systems](https://mlsanity.com/paper/1808.09413/) (2018)
- [Model-based Exploration of the Frontier of Behaviours for Deep Learning System Testing](https://mlsanity.com/paper/2007.02787/) (2020)
- [DeepSmartFuzzer: Reward Guided Test Generation For Deep Learning](https://mlsanity.com/paper/1911.10621/) (2019)

## Built on by (most cited first, 29 of 29)

- [Black-Box Testing of Deep Neural Networks Through Test Case Diversity](https://mlsanity.com/paper/2112.12591/) (2021)
- [When and Why Test Generators for Deep Learning Produce Invalid Inputs: an Empirical Study](https://mlsanity.com/paper/2212.11368/) (2022)
- [Testing of Deep Reinforcement Learning Agents with Surrogate Models](https://mlsanity.com/paper/2305.12751/) (2023)
- [Harnessing Neuron Stability to Improve DNN Verification](https://mlsanity.com/paper/2401.14412/) (2024)
- [DeepMetis: Augmenting a Deep Learning Test Set to Increase its Mutation Score](https://mlsanity.com/paper/2109.07514/) (2021)
- [DeepGD: A Multi-Objective Black-Box Test Selection Approach for Deep Neural Networks](https://mlsanity.com/paper/2303.04878/) (2023)
- [Two is Better Than One: Digital Siblings to Improve Autonomous Driving Testing](https://mlsanity.com/paper/2305.08060/) (2023)
- [Black-box Safety Analysis and Retraining of DNNs based on Feature Extraction and Clustering](https://mlsanity.com/paper/2201.05077/) (2022)
- [Requirements Engineering for Machine Learning: A Review and Reflection](https://mlsanity.com/paper/2210.00859/) (2022)
- [Boundary State Generation for Testing and Improvement of Autonomous Driving Systems](https://mlsanity.com/paper/2307.10590/) (2023)
- [Simulator-based explanation and debugging of hazard-triggering events in DNN-based safety-critical systems](https://mlsanity.com/paper/2204.00480/) (2022)
- [LUNA: A Model-Based Universal Analysis Framework for Large Language Models](https://mlsanity.com/paper/2310.14211/) (2023)
- [Evaluating the Impact of Flaky Simulators on Testing Autonomous Driving Systems](https://mlsanity.com/paper/2311.18768/) (2023)
- [Benchmarking Generative AI Models for Deep Learning Test Input Generation](https://mlsanity.com/paper/2412.17652/) (2024)
- [Targeted Deep Learning System Boundary Testing](https://mlsanity.com/paper/2408.06258/) (2024)
- [Reinforcement Learning for Online Testing of Autonomous Driving Systems: a Replication and Extension Study](https://mlsanity.com/paper/2403.13729/) (2024)
- [RBT4DNN: Requirements-based Testing of Neural Networks](https://mlsanity.com/paper/2504.02737/) (2025)
- [Feature Map Testing for Deep Neural Networks](https://mlsanity.com/paper/2307.11563/) (2023)
- [Can Search-Based Testing with Pareto Optimization Effectively Cover Failure-Revealing Test Inputs?](https://mlsanity.com/paper/2410.11769/) (2024)
- [Navigating the Edge with the State-of-the-Art Insights into Corner Case Identification and Generation for Enhanced Autonomous Vehicle Safety](https://mlsanity.com/paper/2503.00077/) (2025)
- [Simulator Ensembles for Trustworthy Autonomous Driving Systems Testing](https://mlsanity.com/paper/2503.08936/) (2025)
- [Learning test generators for cyber-physical systems](https://mlsanity.com/paper/2410.03202/) (2024)
- [NeRF-To-Real Tester: Neural Radiance Fields as Test Image Generators for Vision of Autonomous Systems](https://mlsanity.com/paper/2412.16141/) (2024)
- [GAN-enhanced Simulation-driven DNN Testing in Absence of Ground Truth](https://mlsanity.com/paper/2503.15953/) (2025)
- [CORTEX-AVD: A Framework for CORner Case Testing and EXploration in Autonomous Vehicle Development](https://mlsanity.com/paper/2504.03989/) (2025)
- [TopoMap: A Feature-based Semantic Discriminator of the Topographical Regions in the Test Input Space](https://mlsanity.com/paper/2509.03242/) (2025)
- [HyperNet-Adaptation for Diffusion-Based Test Case Generation](https://mlsanity.com/paper/2601.15041/) (2026)
- [Latent Regularization in Generative Test Input Generation](https://mlsanity.com/paper/2602.15552/) (2026)
- [Latent Anchor-Driven Test Generation for Deep Neural Networks](https://mlsanity.com/paper/2606.04310/) (2026)

## Abstract (arXiv)

Deep Learning (DL) has been successfully applied to a wide range of application domains, including safety-critical ones. Several DL testing approaches have been recently proposed in the literature but none of them aims to assess how different interpretable features of the generated inputs affect the system's behaviour. In this paper, we resort to Illumination Search to find the highest-performing test cases (i.e., misbehaving and closest to misbehaving), spread across the cells of a map representing the feature space of the system. We introduce a methodology that guides the users of our approach in the tasks of identifying and quantifying the dimensions of the feature space for a given domain. We developed DeepHyperion, a search-based tool for DL systems that illuminates, i.e., explores at large, the feature space, by providing developers with an interpretable feature map where automatically generated inputs are placed along with information about the exposed behaviours.