# Mutation-based Fault Localization of Deep Neural Networks

arXiv: https://arxiv.org/abs/2309.05067 · HTML: https://mlsanity.com/paper/2309.05067/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 69 references that are in the map.
- First posted on arXiv 10 Sep 2023.

## Builds on (most cited first, 17 of 17)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [DeepXplore: Automated Whitebox Testing of Deep Learning Systems](https://mlsanity.com/paper/1705.06640/) (2017)
- [Guiding Deep Learning System Testing using Surprise Adequacy](https://mlsanity.com/paper/1808.08444/) (2018)
- [Taxonomy of Real Faults in Deep Learning Systems](https://mlsanity.com/paper/1910.11015/) (2019)
- [DeepMutation: Mutation Testing of Deep Learning Systems](https://mlsanity.com/paper/1805.05206/) (2018)
- [A Comprehensive Study on Deep Learning Bug Characteristics](https://mlsanity.com/paper/1906.01388/) (2019)
- [DeepLocalize: Fault Localization for Deep Neural Networks](https://mlsanity.com/paper/2103.03376/) (2021)
- [Repairing Deep Neural Networks: Fix Patterns and Challenges](https://mlsanity.com/paper/2005.00972/) (2020)
- [DeepFault: Fault Localization for Deep Neural Networks](https://mlsanity.com/paper/1902.05974/) (2019)
- [Practical Program Repair via Bytecode Mutation](https://mlsanity.com/paper/1807.03512/) (2018)
- [DeepDiagnosis: Automatically Diagnosing Faults and Recommending Actionable Fixes in Deep Learning Programs](https://mlsanity.com/paper/2112.04036/) (2021)
- [DeepFD: Automated Fault Diagnosis and Localization for Deep Learning Programs](https://mlsanity.com/paper/2205.01938/) (2022)
- [Bugs in Machine Learning-based Systems: A Faultload Benchmark](https://mlsanity.com/paper/2206.12311/) (2022)
- [Automatic Fault Detection for Deep Learning Programs Using Graph Transformations](https://mlsanity.com/paper/2105.08095/) (2021)
- [Empirical Evaluation of Mutation-based Test Prioritization Techniques](https://mlsanity.com/paper/1709.04631/) (2017)
- [Inferring Loop Invariants by Mutation, Dynamic Analysis, and Static Checking](https://mlsanity.com/paper/1407.5286/) (2014)
- [A Theoretical Framework for Understanding Mutation-Based Testing Methods](https://mlsanity.com/paper/1601.06466/) (2016)

## Built on by (most cited first, 5 of 5)

- [BLAZE: Cross-Language and Cross-Project Bug Localization via Dynamic Chunking and Hard Example Learning](https://mlsanity.com/paper/2407.17631/) (2024)
- [Debugging and Runtime Analysis of Neural Networks with VLMs (A Case Study)](https://mlsanity.com/paper/2503.17416/) (2025)
- [Feature-Guided Analysis of Neural Networks: A Replication Study](https://mlsanity.com/paper/2511.00052/) (2025)
- [On Accelerating Deep Neural Network Mutation Analysis by Neuron and Mutant Clustering](https://mlsanity.com/paper/2501.12598/) (2025)
- [Path Analysis for Effective Fault Localization in Deep Neural Networks](https://mlsanity.com/paper/2310.18987/) (2023)

## Abstract (arXiv)

Deep neural networks (DNNs) are susceptible to bugs, just like other types of software systems. A significant uptick in using DNN, and its applications in wide-ranging areas, including safety-critical systems, warrant extensive research on software engineering tools for improving the reliability of DNN-based systems. One such tool that has gained significant attention in the recent years is DNN fault localization. This paper revisits mutation-based fault localization in the context of DNN models and proposes a novel technique, named deepmufl, applicable to a wide range of DNN models. We have implemented deepmufl and have evaluated its effectiveness using 109 bugs obtained from StackOverflow. Our results show that deepmufl detects 53/109 of the bugs by ranking the buggy layer in top-1 position, outperforming state-of-the-art static and dynamic DNN fault localization systems that are also designed to target the class of bugs supported by deepmufl. Moreover, we observed that we can halve the fault localization time for a pre-trained model using mutation selection, yet losing only 7.55% of the bugs localized in top-1 position.