mlsanityData to 14 Oct 2026

TensorFuzz: Debugging Neural Networks with Coverage-Guided Fuzzing

arXiv 1807.10875 ↗First posted 28 Jul 2018stat.ML cs.LGcited by 68

Not on the map. Why?

Abstract (arXiv)

Machine learning models are notoriously difficult to interpret and debug. This is particularly true of neural networks. In this work, we introduce automated software testing techniques for neural networks that are well-suited to discovering errors which occur only for rare inputs. Specifically, we develop coverage-guided fuzzing (CGF) methods for neural networks. In CGF, random mutations of inputs to a neural network are guided by a coverage metric toward the goal of satisfying user-specified constraints. We describe how fast approximate nearest neighbor algorithms can provide this coverage metric. We then discuss the application of CGF to the following goals: finding numerical errors in trained neural networks, generating disagreements between neural networks and quantized versions of those networks, and surfacing undesirable behavior in character level language models. Finally, we release an open source library called TensorFuzz that implements the described techniques.

Key facts. Cited by 68 papers in the map (data through 14 Oct 2026). First posted on arXiv 28 Jul 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 16; 2022: 16; 2023: 15; 2024: 7; 2025: 7; 2026: 7.

68 papers.

Sort:

By area:

  1. 2021Software Engineering for AI-Based Systems: A Surveycited by 69
  2. 2021Provable Repair of Deep Neural Networkscited by 42
  3. 2021Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safetycited by 35
  4. 2021ModelDiff: Testing-Based DNN Similarity Comparison for Model Reuse Detectioncited by 33
  5. 2021Distribution-Aware Testing of Neural Networks Using Generative Modelscited by 22
  6. 2022NPC: Neuron Path Coverage via Characterizing Decision Logic of Deep Neural Networkscited by 18
  7. 2021Understanding Performance Problems in Deep Learning Systemscited by 12
  8. 2021MDPFuzz: Testing Models Solving Markov Decision Processescited by 12
  9. 2022QVIP: An ILP-based Formal Verification Approach for Quantized Neural Networkscited by 12
  10. 2025Neural Network Verification is a Programming Language Challengecited by 12
  11. 2021Revisiting Neuron Coverage for DNN Testing: A Layer-Wise and Distribution-Aware Criterioncited by 11
  12. 2022Coverage-Guided Tensor Compiler Fuzzing with Joint IR-Pass Mutationcited by 10
38 more
  1. 2024A Survey on Verification and Validation, Testing and Evaluations of Neurosymbolic Artificial Intelligencecited by 10
  2. 2021AequeVox: Automated Fairness Testing of Speech Recognition Systemscited by 9
  3. 2022QEBVerif: Quantization Error Bound Verification of Neural Networkscited by 9
  4. 2023Preemptively Pruning Clever-Hans Strategies in Deep Neural Networkscited by 9
  5. 2026ChainFuzzer: Greybox Fuzzing for Workflow-Level Multi-Tool Vulnerabilities in LLM Agentscited by 9
  6. 2022Generating and Detecting True Ambiguity: A Forgotten Danger in DNN Supervision Testingcited by 8
  7. 2023Machine Learning with Requirements: a Manifestocited by 8
  8. 2024Machine Learning Robustness: A Primercited by 8
  9. 2024Understanding the Effectiveness of Coverage Criteria for Large Language Models: A Special Angle from Jailbreak Attackscited by 8
  10. 2021ArchRepair: Block-Level Architecture-Oriented Repairing for Deep Neural Networkscited by 7
  11. 2021Distribution Awareness for AI System Testingcited by 6
  12. 2023FedDebug: Systematic Debugging for Federated Learning Applicationscited by 6
  13. 2021Active Assessment of Prediction Services as Accuracy Surface Over Attribute Combinationscited by 5
  14. 2022Software Testing for Machine Learningcited by 5
  15. 2022COMET: Coverage-guided Model Generation For Deep Learning Library Testingcited by 4
  16. 2023Exploring Adversarial Attacks on Neural Networks: An Explainable Approachcited by 4
  17. 2023Test & Evaluation Best Practices for Machine Learning-Enabled Systemscited by 4
  18. 2022Tailored Uncertainty Estimation for Deep Learning Systemscited by 3
  19. 2022DiverGet: A Search-Based Software Testing Approach for Deep Neural Network Quantization Assessmentcited by 3
  20. 2023Reliability Assurance for Deep Neural Network Architectures Against Numerical Defectscited by 3
  21. 2023Semantic-Based Neural Network Repaircited by 3
  22. 2023NeuroCodeBench: a plain C neural network benchmark for software verificationcited by 3
  23. 2024Automated Repair of AI Code with Large Language Models and Formal Verificationcited by 3
  24. 2021ML4ML: Automated Invariance Testing for Machine Learning Modelscited by 2
  25. 2022Testing Feedforward Neural Networks Training Programscited by 2
  26. 2023NoiseCAM: Explainable AI for the Boundary Between Noise and Adversarial Attackscited by 2
  27. 2025Adaptive Branch-and-Bound Tree Exploration for Neural Network Verificationcited by 2
  28. 2021Defuse: Harnessing Unrestricted Adversarial Examples for Debugging Models Beyond Test Accuracycited by 1
  29. 2021Provably Valid and Diverse Mutations of Real-World Media Data for DNN Testingcited by 1
  30. 2021Finding Deviated Behaviors of the Compressed DNN Models for Image Classificationscited by 1
  31. 2022LaF: Labeling-Free Model Selection for Automated Deep Neural Network Reusingcited by 1
  32. 2022Is Neuron Coverage Needed to Make Person Detection More Robust?cited by 1
  33. 2023Feature Map Testing for Deep Neural Networkscited by 1
  34. 2023DREAM: Debugging and Repairing AutoML Pipelinescited by 1
  35. 2024Precision Guided Approach to Mitigate Data Poisoning Attacks in Federated Learningcited by 1
  36. 2026Content Fuzzing for Escaping Information Cocoons on Digital Social Mediacited by 1
  37. 2022Safe AI -- How is this Possible?cited by 0
  38. 2022Excitement Surfeited Turns to Errors: Deep Learning Testing Framework Based on Excitable Neuronscited by 0

50 most cited of 68 All 68 (JSON)