# DeepDiagnosis: Automatically Diagnosing Faults and Recommending Actionable Fixes in Deep Learning Programs arXiv: https://arxiv.org/abs/2112.04036 ยท HTML: https://mlsanity.com/paper/2112.04036/ ## Key facts - Cited by 19 papers in the map (data through 14 Oct 2026). - Builds on 15 of its 41 references that are in the map. - First posted on arXiv 7 Dec 2021. ## Builds on (most cited first, 15 of 15) - [DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars](https://mlsanity.com/paper/1708.08559/) (2017) - [Stable Recurrent Models](https://mlsanity.com/paper/1805.10369/) (2018) - [A Comprehensive Study on Deep Learning Bug Characteristics](https://mlsanity.com/paper/1906.01388/) (2019) - [Noisy Activation Functions](https://mlsanity.com/paper/1603.00391/) (2016) - [Do the Machine Learning Models on a Crowd Sourced Platform Exhibit Bias? An Empirical Study on Model Fairness](https://mlsanity.com/paper/2005.12379/) (2020) - [Fair Preprocessing: Towards Understanding Compositional Fairness of Data Transformers in Machine Learning Pipeline](https://mlsanity.com/paper/2106.06054/) (2021) - [DeepLocalize: Fault Localization for Deep Neural Networks](https://mlsanity.com/paper/2103.03376/) (2021) - [Repairing Deep Neural Networks: Fix Patterns and Challenges](https://mlsanity.com/paper/2005.00972/) (2020) - [DeepFault: Fault Localization for Deep Neural Networks](https://mlsanity.com/paper/1902.05974/) (2019) - [The Art and Practice of Data Science Pipelines: A Comprehensive Study of Data Science Pipelines In Theory, In-The-Small, and In-The-Large](https://mlsanity.com/paper/2112.01590/) (2021) - [Random Walk Initialization for Training Very Deep Feedforward Networks](https://mlsanity.com/paper/1412.6558/) (2014) - [Revise Saturated Activation Functions](https://mlsanity.com/paper/1602.05980/) (2016) - [Detecting Dead Weights and Units in Neural Networks](https://mlsanity.com/paper/1806.06068/) (2018) - [The effect of Target Normalization and Momentum on Dying ReLU](https://mlsanity.com/paper/2005.06195/) (2020) - [Manas: Mining Software Repositories to Assist AutoML](https://mlsanity.com/paper/2112.03395/) (2021) ## Built on by (most cited first, 19 of 19) - [Bugs in Machine Learning-based Systems: A Faultload Benchmark](https://mlsanity.com/paper/2206.12311/) (2022) - [Bug Characterization in Machine Learning-based Systems](https://mlsanity.com/paper/2307.14512/) (2023) - [Mutation-based Fault Localization of Deep Neural Networks](https://mlsanity.com/paper/2309.05067/) (2023) - [Semantic-Based Neural Network Repair](https://mlsanity.com/paper/2306.07995/) (2023) - [Debugging and Runtime Analysis of Neural Networks with VLMs (A Case Study)](https://mlsanity.com/paper/2503.17416/) (2025) - [Feature-Guided Analysis of Neural Networks: A Replication Study](https://mlsanity.com/paper/2511.00052/) (2025) - [Toward Debugging Deep Reinforcement Learning Programs with RLExplorer](https://mlsanity.com/paper/2410.04322/) (2024) - [IRepair: An Intent-Aware Approach to Repair Data-Driven Errors in Large Language Models](https://mlsanity.com/paper/2502.07072/) (2025) - [An Experience Report on Regression-Free Repair of Deep Neural Network Model](https://mlsanity.com/paper/2503.07079/) (2025) - [Why Attention Fails: A Taxonomy of Faults in Attention-Based Neural Networks](https://mlsanity.com/paper/2508.04925/) (2025) - [A Retrospective on ICSE 2022](https://mlsanity.com/paper/2207.12578/) (2022) - [Comparative analysis of real bugs in open-source Machine Learning projects -- A Registered Report](https://mlsanity.com/paper/2209.09932/) (2022) - [Path Analysis for Effective Fault Localization in Deep Neural Networks](https://mlsanity.com/paper/2310.18987/) (2023) - [An Empirical Study of Fault Localisation Techniques for Deep Learning](https://mlsanity.com/paper/2412.11304/) (2024) - [GAN-enhanced Simulation-driven DNN Testing in Absence of Ground Truth](https://mlsanity.com/paper/2503.15953/) (2025) - [BTTackler: A Diagnosis-based Framework for Efficient Deep Learning Hyperparameter Optimization](https://mlsanity.com/paper/2602.23630/) (2026) - [Hierarchical Fault Detection and Diagnosis for Transformer Architectures](https://mlsanity.com/paper/2604.28118/) (2026) - [Evaluation-Strategy Gap in Fault Diagnosis of Deep Learning Programs](https://mlsanity.com/paper/2606.26492/) (2026) - [Deep4ge: DNN Training Trajectories for Fault Detection and Diagnosis](https://mlsanity.com/paper/2607.12868/) (2026) ## Abstract (arXiv) Deep Neural Networks (DNNs) are used in a wide variety of applications. However, as in any software application, DNN-based apps are afflicted with bugs. Previous work observed that DNN bug fix patterns are different from traditional bug fix patterns. Furthermore, those buggy models are non-trivial to diagnose and fix due to inexplicit errors with several options to fix them. To support developers in locating and fixing bugs, we propose DeepDiagnosis, a novel debugging approach that localizes the faults, reports error symptoms and suggests fixes for DNN programs. In the first phase, our technique monitors a training model, periodically checking for eight types of error conditions. Then, in case of problems, it reports messages containing sufficient information to perform actionable repairs to the model. In the evaluation, we thoroughly examine 444 models -53 real-world from GitHub and Stack Overflow, and 391 curated by AUTOTRAINER. DeepDiagnosis provides superior accuracy when compared to UMLUAT and DeepLocalize. Our technique is faster than AUTOTRAINER for fault localization. The results show that our approach can support additional types of models, while state-of-the-art was only able to handle classification ones. Our technique was able to report bugs that do not manifest as numerical errors during training. Also, it can provide actionable insights for fix whereas DeepLocalize can only report faults that lead to numerical errors during training. DeepDiagnosis manifests the best capabilities of fault detection, bug localization, and symptoms identification when compared to other approaches.