mlsanityData to 7 Oct 2026

DeepHyperion: Exploring the Feature Space of Deep Learning-Based Systems through Illumination Search

arXiv 2107.06997 ↗First posted 5 Jul 2021cs.LG cs.AI cs.SEcited by 29

Not on the map. Why?

Abstract (arXiv)

Deep Learning (DL) has been successfully applied to a wide range of application domains, including safety-critical ones. Several DL testing approaches have been recently proposed in the literature but none of them aims to assess how different interpretable features of the generated inputs affect the system's behaviour. In this paper, we resort to Illumination Search to find the highest-performing test cases (i.e., misbehaving and closest to misbehaving), spread across the cells of a map representing the feature space of the system. We introduce a methodology that guides the users of our approach in the tasks of identifying and quantifying the dimensions of the feature space for a given domain. We developed DeepHyperion, a search-based tool for DL systems that illuminates, i.e., explores at large, the feature space, by providing developers with an interpretable feature map where automatically generated inputs are placed along with information about the exposed behaviours.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Builds on 12 of its 41 references that are in the map. First posted on arXiv 5 Jul 2021.

Builds on

12 of 41 references.

References · 12

  1. 2016End to End Learning for Self-Driving Carscited by 1,000
  2. 2015Illuminating search spaces by mapping elitescited by 406
  3. 2017DeepXplore: Automated Whitebox Testing of Deep Learning Systemscited by 307
  4. 2017DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Carscited by 249
  5. 2016Data Programming: Creating Large Training Sets, Quicklycited by 246
  6. 2019Machine Learning Testing: Survey, Landscapes and Horizonscited by 213
  7. 2018DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systemscited by 144
  8. 2018Guiding Deep Learning System Testing using Surprise Adequacycited by 123
  9. 2019Taxonomy of Real Faults in Deep Learning Systemscited by 90
  10. 2018DLFuzz: Differential Fuzzing Testing of Deep Learning Systemscited by 64
  11. 2020Model-based Exploration of the Frontier of Behaviours for Deep Learning System Testingcited by 40
  12. 2019DeepSmartFuzzer: Reward Guided Test Generation For Deep Learningcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 4; 2023: 7; 2024: 7; 2025: 6; 2026: 3.

29 papers.

Sort:

By area:

  1. 2021Black-Box Testing of Deep Neural Networks Through Test Case Diversitycited by 18
  2. 2022When and Why Test Generators for Deep Learning Produce Invalid Inputs: an Empirical Studycited by 18
  3. 2024Harnessing Neuron Stability to Improve DNN Verificationcited by 13
  4. 2021DeepMetis: Augmenting a Deep Learning Test Set to Increase its Mutation Scorecited by 12
  5. 2023Testing of Deep Reinforcement Learning Agents with Surrogate Modelscited by 12
  6. 2023DeepGD: A Multi-Objective Black-Box Test Selection Approach for Deep Neural Networkscited by 11
  7. 2022Black-box Safety Analysis and Retraining of DNNs based on Feature Extraction and Clusteringcited by 8
  8. 2023Two is Better Than One: Digital Siblings to Improve Autonomous Driving Testingcited by 8
  9. 2022Requirements Engineering for Machine Learning: A Review and Reflectioncited by 7
  10. 2023Boundary State Generation for Testing and Improvement of Autonomous Driving Systemscited by 6
  11. 2022Simulator-based explanation and debugging of hazard-triggering events in DNN-based safety-critical systemscited by 4
  12. 2023LUNA: A Model-Based Universal Analysis Framework for Large Language Modelscited by 4
17 more
  1. 2023Evaluating the Impact of Flaky Simulators on Testing Autonomous Driving Systemscited by 4
  2. 2024Benchmarking Generative AI Models for Deep Learning Test Input Generationcited by 4
  3. 2024Targeted Deep Learning System Boundary Testingcited by 3
  4. 2024Reinforcement Learning for Online Testing of Autonomous Driving Systems: a Replication and Extension Studycited by 2
  5. 2025RBT4DNN: Requirements-based Testing of Neural Networkscited by 2
  6. 2023Feature Map Testing for Deep Neural Networkscited by 1
  7. 2024Can Search-Based Testing with Pareto Optimization Effectively Cover Failure-Revealing Test Inputs?cited by 1
  8. 2025Navigating the Edge with the State-of-the-Art Insights into Corner Case Identification and Generation for Enhanced Autonomous Vehicle Safetycited by 1
  9. 2025Simulator Ensembles for Trustworthy Autonomous Driving Systems Testingcited by 1
  10. 2024Learning test generators for cyber-physical systemscited by 0
  11. 2024NeRF-To-Real Tester: Neural Radiance Fields as Test Image Generators for Vision of Autonomous Systemscited by 0
  12. 2025GAN-enhanced Simulation-driven DNN Testing in Absence of Ground Truthcited by 0
  13. 2025CORTEX-AVD: A Framework for CORner Case Testing and EXploration in Autonomous Vehicle Developmentcited by 0
  14. 2025TopoMap: A Feature-based Semantic Discriminator of the Topographical Regions in the Test Input Spacecited by 0
  15. 2026HyperNet-Adaptation for Diffusion-Based Test Case Generationcited by 0
  16. 2026Latent Regularization in Generative Test Input Generationcited by 0
  17. 2026Latent Anchor-Driven Test Generation for Deep Neural Networkscited by 0