mlsanityData to 7 Oct 2026

DeepGD: A Multi-Objective Black-Box Test Selection Approach for Deep Neural Networks

arXiv 2303.04878 ↗First posted 8 Mar 2023cs.LG cs.PF cs.SEcited by 11

Not on the map. Why?

Abstract (arXiv)

Deep neural networks (DNNs) are widely used in various application domains such as image processing, speech recognition, and natural language processing. However, testing DNN models may be challenging due to the complexity and size of their input domain. Particularly, testing DNN models often requires generating or exploring large unlabeled datasets. In practice, DNN test oracles, which identify the correct outputs for inputs, often require expensive manual effort to label test data, possibly involving multiple experts to ensure labeling correctness. In this paper, we propose DeepGD, a black-box multi-objective test selection approach for DNN models. It reduces the cost of labeling by prioritizing the selection of test inputs with high fault revealing power from large unlabeled datasets. DeepGD not only selects test inputs with high uncertainty scores to trigger as many mispredicted inputs as possible but also maximizes the probability of revealing distinct faults in the DNN model by selecting diverse mispredicted inputs. The experimental results conducted on four widely used datasets and five DNN models show that in terms of fault-revealing ability: (1) White-box, coverage-based approaches fare poorly, (2) DeepGD outperforms existing black-box test selection approaches in terms of fault detection, and (3) DeepGD also leads to better guidance for DNN model retraining when using selected inputs to augment the training set.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 28 of its 76 references that are in the map. First posted on arXiv 8 Mar 2023.

Builds on

28 of 76 references.

References · 28

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  2. 2017A Survey on Deep Learning in Medical Image Analysiscited by 1,182
  3. 2019Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Boundscited by 602
  4. 2019A Survey of Deep Learning Techniques for Autonomous Drivingcited by 401
  5. 2017DeepXplore: Automated Whitebox Testing of Deep Learning Systemscited by 307
  6. 2017DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Carscited by 249
  7. 2020pymoo: Multi-objective Optimization in Pythoncited by 196
  8. 2018DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systemscited by 144
  9. 2018Guiding Deep Learning System Testing using Surprise Adequacycited by 123
  10. 2019Deep Active Learning: Unified and Principled Method for Query and Trainingcited by 92
  11. 2022A Comparative Survey of Deep Active Learningcited by 88
  12. 2018Diversity in Machine Learningcited by 48
16 more
  1. 2019Survey on Deep Neural Networks in Speech and Vision Systemscited by 37
  2. 2018Testing Deep Neural Networkscited by 36
  3. 2021How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Reviewcited by 36
  4. 2021DeepHyperion: Exploring the Feature Space of Deep Learning-Based Systems through Illumination Searchcited by 29
  5. 2019Test Selection for Deep Learning Systemscited by 27
  6. 2022Simple Techniques Work Surprisingly Well for Neural Network Test Prioritization and Active Learning (Replicability Study)cited by 27
  7. 2020Importance-Driven Deep Learning System Testingcited by 24
  8. 2019Boosting Operational DNN Testing Efficiency through Conditioningcited by 22
  9. 2021Black-Box Testing of Deep Neural Networks Through Test Case Diversitycited by 18
  10. 2022Revisiting Neuron Coverage Metrics and Quality of Deep Neural Networkscited by 14
  11. 2021Test Case Selection and Prioritization Using Machine Learning: A Systematic Literature Reviewcited by 12
  12. 2022A Search-Based Testing Approach for Deep Reinforcement Learning Agentscited by 12
  13. 2020Supporting DNN Safety Analysis and Retraining through Heatmap-based Unsupervised Learningcited by 11
  14. 2015Test Set Diameter: Quantifying the Diversity of Sets of Test Casescited by 10
  15. 2022Black-box Safety Analysis and Retraining of DNNs based on Feature Extraction and Clusteringcited by 8
  16. 2022ATM: Black-box Test Case Minimization based on Test Code Similarity and Evolutionary Searchcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 3; 2025: 4; 2026: 3.

11 papers.

  1. 2024VLATest: Testing and Evaluating Vision-Language-Action Models for Robotic Manipulationcited by 43
  2. 2023SMARLA: A Safety Monitoring Approach for Deep Reinforcement Learning Agentscited by 9
  3. 2025MetaSel: A Test Selection Approach for Fine-tuned DNN Modelscited by 3
  4. 2026A Highly Efficient Diversity-based Input Selection for DNN Improvement Using VLMscited by 1
  5. 2024Robust Black-box Testing of Deep Neural Networks using Co-Domain Coveragecited by 0
  6. 2024DiffGAN: A Test Generation Approach for Differential Testing of Deep Neural Networks for Image Analysiscited by 0
  7. 2025TopoMap: A Feature-based Semantic Discriminator of the Topographical Regions in the Test Input Spacecited by 0
  8. 2025Learning-Based Testing for Deep Learning: Enhancing Model Robustness with Adversarial Input Prioritizationcited by 0
  9. 2025Bias Testing and Mitigation in Black Box LLMs using Metamorphic Relationscited by 0
  10. 2026CAFD: Concept-Aware DNN Fault Detection using VLMscited by 0
  11. 2026Learnable Weighting of Intra-Attribute Distances for Categorical Data Clustering with Nominal and Ordinal Attributescited by 0