mlsanityData to 7 Oct 2026

Testing Deep Neural Networks

arXiv 1803.04792 ↗First posted 10 Mar 2018cs.LG cs.CV cs.SEcited by 36

Not on the map. Why?

Abstract (arXiv)

Deep neural networks (DNNs) have a wide range of applications, and software employing them must be thoroughly tested, especially in safety-critical domains. However, traditional software test coverage metrics cannot be applied directly to DNNs. In this paper, inspired by the MC/DC coverage criterion, we propose a family of four novel test criteria that are tailored to structural features of DNNs and their semantics. We validate the criteria by demonstrating that the generated test inputs guided via our proposed coverage criteria are able to capture undesired behaviours in a DNN. Test cases are generated using a symbolic approach and a gradient-based heuristic search. By comparing them with existing methods, we show that our criteria achieve a balance between their ability to find bugs (proxied using adversarial examples) and the computational cost of test case generation. Our experiments are conducted on state-of-the-art DNNs obtained using popular open source datasets, including MNIST, CIFAR-10 and ImageNet.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 5; 2023: 8; 2024: 6; 2025: 3; 2026: 2.

36 papers.

Sort:

By area:

  1. 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 91
  2. 2021Deep Ensembles from a Bayesian Perspectivecited by 27
  3. 2021Distribution-Aware Testing of Neural Networks Using Generative Modelscited by 22
  4. 2021Taxonomy of Machine Learning Safety: A Survey and Primercited by 22
  5. 2021Attack as Defense: Characterizing Adversarial Examples using Robustnesscited by 14
  6. 2021Minimal Multi-Layer Modifications of Deep Neural Networkscited by 14
  7. 2022A Search-Based Testing Approach for Deep Reinforcement Learning Agentscited by 12
  8. 2021MDPFuzz: Testing Models Solving Markov Decision Processescited by 11
  9. 2023DeepGD: A Multi-Objective Black-Box Test Selection Approach for Deep Neural Networkscited by 11
  10. 2021Increasing the Confidence of Deep Neural Networks by Coverage Analysiscited by 9
  11. 2021Revisiting Neuron Coverage for DNN Testing: A Layer-Wise and Distribution-Aware Criterioncited by 9
  12. 2021Customizable Reference Runtime Monitoring of Neural Networks using Resolution Boxescited by 6
24 more
  1. 2021Fairness Testing of Deep Image Classification with Adequacy Metricscited by 6
  2. 2022Software Testing for Machine Learningcited by 5
  3. 2024Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forwardcited by 5
  4. 2021Corner case data description and detectioncited by 4
  5. 2023LUNA: A Model-Based Universal Analysis Framework for Large Language Modelscited by 4
  6. 2023Neuron Sensitivity Guided Test Case Selection for Deep Learning Testingcited by 3
  7. 2024Targeted Deep Learning System Boundary Testingcited by 3
  8. 2024A Survey of Neural Network Robustness Assessment in Image Recognitioncited by 2
  9. 2021Abstraction and Symbolic Execution of Deep Neural Networks with Bayesian Approximation of Hidden Featurescited by 1
  10. 2022Fixed-Point Code Synthesis For Neural Networkscited by 1
  11. 2022Is Neuron Coverage Needed to Make Person Detection More Robust?cited by 1
  12. 2023D-Score: A White-Box Diagnosis Score for CNNs Based on Mutation Operatorscited by 1
  13. 2023Feature Map Testing for Deep Neural Networkscited by 1
  14. 2022Safe AI -- How is this Possible?cited by 0
  15. 2023Testing the Channels of Convolutional Neural Networkscited by 0
  16. 2023Path Analysis for Effective Fault Localization in Deep Neural Networkscited by 0
  17. 2024Beyond Accuracy: An Empirical Study on Unit Testing in Open-source Deep Learning Projectscited by 0
  18. 2024DeepKnowledge: Generalisation-Driven Deep Learning Testingcited by 0
  19. 2024LeCov: Multi-level Testing Criteria for Large Language Modelscited by 0
  20. 2025Towards Understanding Deep Learning Model in Image Recognition via Coverage Testcited by 0
  21. 2025A Topological Improvement of the Overall Performance of Sparse Evolutionary Training: Motif-Based Structural Optimization of Sparse MLPs Projectcited by 0
  22. 2025Influence-Guided Concolic Testing of Transformer Robustnesscited by 0
  23. 2026ADEPT: A Unified Framework for Deep Learning Test Adequacycited by 0
  24. 2026Towards Systematic Qualification of Vision-Language Models for Automotive Perception Systemscited by 0