# Repairing Deep Neural Networks: Fix Patterns and Challenges

arXiv: https://arxiv.org/abs/2005.00972 · HTML: https://mlsanity.com/paper/2005.00972/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 May 2020.

## Built on by (most cited first, 30 of 33)

- [Fair Preprocessing: Towards Understanding Compositional Fairness of Data Transformers in Machine Learning Pipeline](https://mlsanity.com/paper/2106.06054/) (2021)
- [NNrepair: Constraint-based Repair of Neural Network Classifiers](https://mlsanity.com/paper/2103.12535/) (2021)
- [An Empirical Study on Deployment Faults of Deep Learning Based Mobile Applications](https://mlsanity.com/paper/2101.04930/) (2021)
- [Neural Network Repair with Reachability Analysis](https://mlsanity.com/paper/2108.04214/) (2021)
- [DeepDiagnosis: Automatically Diagnosing Faults and Recommending Actionable Fixes in Deep Learning Programs](https://mlsanity.com/paper/2112.04036/) (2021)
- [Faults in Deep Reinforcement Learning Programs: A Taxonomy and A Detection Approach](https://mlsanity.com/paper/2101.00135/) (2021)
- [DeepFD: Automated Fault Diagnosis and Localization for Deep Learning Programs](https://mlsanity.com/paper/2205.01938/) (2022)
- [Bugs in Machine Learning-based Systems: A Faultload Benchmark](https://mlsanity.com/paper/2206.12311/) (2022)
- [Understanding Performance Problems in Deep Learning Systems](https://mlsanity.com/paper/2112.01771/) (2021)
- [Challenges and Practices of Deep Learning Model Reengineering: A Case Study on Computer Vision](https://mlsanity.com/paper/2303.07476/) (2023)
- [Design Smells in Deep Learning Programs: An Empirical Study](https://mlsanity.com/paper/2107.02279/) (2021)
- [Bug Characterization in Machine Learning-based Systems](https://mlsanity.com/paper/2307.14512/) (2023)
- [Automatic Fault Detection for Deep Learning Programs Using Graph Transformations](https://mlsanity.com/paper/2105.08095/) (2021)
- [Manas: Mining Software Repositories to Assist AutoML](https://mlsanity.com/paper/2112.03395/) (2021)
- [OccRob: Efficient SMT-Based Occlusion Robustness Verification of Deep Neural Networks](https://mlsanity.com/paper/2301.11912/) (2023)
- [Mutation-based Fault Localization of Deep Neural Networks](https://mlsanity.com/paper/2309.05067/) (2023)
- [NEUROSPF: A tool for the Symbolic Analysis of Neural Networks](https://mlsanity.com/paper/2103.00124/) (2021)
- [Test & Evaluation Best Practices for Machine Learning-Enabled Systems](https://mlsanity.com/paper/2310.06800/) (2023)
- [Semantic-Based Neural Network Repair](https://mlsanity.com/paper/2306.07995/) (2023)
- [FetaFix: Automatic Fault Localization and Repair of Deep Learning Model Conversions](https://mlsanity.com/paper/2312.15101/) (2023)
- [Challenges in Migrating Imperative Deep Learning Programs to Graph Execution: An Empirical Study](https://mlsanity.com/paper/2201.09953/) (2022)
- [What is Software Quality for AI Engineers? Towards a Thinning of the Fog](https://mlsanity.com/paper/2203.12697/) (2022)
- [Testing Feedforward Neural Networks Training Programs](https://mlsanity.com/paper/2204.00694/) (2022)
- [What Kinds of Contracts Do ML APIs Need?](https://mlsanity.com/paper/2307.14465/) (2023)
- [Unraveling Code Clone Dynamics in Deep Learning Frameworks](https://mlsanity.com/paper/2404.17046/) (2024)
- [Enabling Automatic Repair of Source Code Vulnerabilities Using Data-Driven Methods](https://mlsanity.com/paper/2202.03055/) (2022)
- [Inferring Data Preconditions from Deep Learning Models for Trustworthy Prediction in Deployment](https://mlsanity.com/paper/2401.14628/) (2024)
- [Repairing Group-Level Errors for DNNs Using Weighted Regularization](https://mlsanity.com/paper/2203.13612/) (2022)
- [Causal Repair of Learning-enabled Cyber-physical Systems](https://mlsanity.com/paper/2304.02813/) (2023)
- [Beyond Accuracy: An Empirical Study on Unit Testing in Open-source Deep Learning Projects](https://mlsanity.com/paper/2402.16546/) (2024)

## Abstract (arXiv)

Significant interest in applying Deep Neural Network (DNN) has fueled the need to support engineering of software that uses DNNs. Repairing software that uses DNNs is one such unmistakable SE need where automated tools could be beneficial; however, we do not fully understand challenges to repairing and patterns that are utilized when manually repairing DNNs. What challenges should automated repair tools address? What are the repair patterns whose automation could help developers? Which repair patterns should be assigned a higher priority for building automated bug repair tools? This work presents a comprehensive study of bug fix patterns to address these questions. We have studied 415 repairs from Stack overflow and 555 repairs from Github for five popular deep learning libraries Caffe, Keras, Tensorflow, Theano, and Torch to understand challenges in repairs and bug repair patterns. Our key findings reveal that DNN bug fix patterns are distinctive compared to traditional bug fix patterns; the most common bug fix patterns are fixing data dimension and neural network connectivity; DNN bug fixes have the potential to introduce adversarial vulnerabilities; DNN bug fixes frequently introduce new bugs; and DNN bug localization, reuse of trained model, and coping with frequent releases are major challenges faced by developers when fixing bugs. We also contribute a benchmark of 667 DNN (bug, repair) instances.