# On Interpretability of Artificial Neural Networks: A Survey

arXiv: https://arxiv.org/abs/2001.02522 · HTML: https://mlsanity.com/paper/2001.02522/

## Key facts

- Cited by 88 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jan 2020.

## Built on by (most cited first, 30 of 88)

- [Explainable artificial intelligence (XAI) in deep learning-based medical image analysis](https://mlsanity.com/paper/2107.10912/) (2021)
- [Explainable AI (XAI): A Systematic Meta-Survey of Current Challenges and Future Opportunities](https://mlsanity.com/paper/2111.06420/) (2021)
- [Wav-KAN: Wavelet Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2405.12832/) (2024)
- [Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2412.02104/) (2024)
- [CTformer: Convolution-free Token2Token Dilated Vision Transformer for Low-dose CT Denoising](https://mlsanity.com/paper/2202.13517/) (2022)
- [Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Direction](https://mlsanity.com/paper/2409.00265/) (2024)
- [Gradient-based Parameter Selection for Efficient Fine-Tuning](https://mlsanity.com/paper/2312.10136/) (2023)
- [The Probabilities Also Matter: A More Faithful Metric for Faithfulness of Free-Text Explanations in Large Language Models](https://mlsanity.com/paper/2404.03189/) (2024)
- [Counterfactual Learning on Graphs: A Survey](https://mlsanity.com/paper/2304.01391/) (2023)
- [Understanding and Controlling a Maze-Solving Policy Network](https://mlsanity.com/paper/2310.08043/) (2023)
- [Interpretable deep-learning models to help achieve the Sustainable Development Goals](https://mlsanity.com/paper/2108.10744/) (2021)
- [Knowledge-tuning Large Language Models with Structured Medical Knowledge Bases for Reliable Response Generation in Chinese](https://mlsanity.com/paper/2309.04175/) (2023)
- [Compute Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders](https://mlsanity.com/paper/2411.13117/) (2024)
- [CausalFormer: An Interpretable Transformer for Temporal Causal Discovery](https://mlsanity.com/paper/2406.16708/) (2024)
- [From superposition to sparse codes: interpretable representations in neural networks](https://mlsanity.com/paper/2503.01824/) (2025)
- [On Recurrent Neural Networks for learning-based control: recent results and ideas for future developments](https://mlsanity.com/paper/2111.13557/) (2021)
- [Endowing Interpretability for Neural Cognitive Diagnosis by Efficient Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2405.14399/) (2024)
- [Interpretability of Machine Learning: Recent Advances and Future Prospects](https://mlsanity.com/paper/2305.00537/) (2023)
- [Unsupervised Deep Learning for IoT Time Series](https://mlsanity.com/paper/2302.03284/) (2023)
- [Explainable AI (XAI) in Image Segmentation in Medicine, Industry, and Beyond: A Survey](https://mlsanity.com/paper/2405.01636/) (2024)
- [Explainability in Practice: A Survey of Explainable NLP Across Various Domains](https://mlsanity.com/paper/2502.00837/) (2025)
- [Discovering interpretable elastoplasticity models via the neural polynomial method enabled symbolic regressions](https://mlsanity.com/paper/2307.13149/) (2023)
- [Instruct Large Language Models to Generate Scientific Literature Survey Step by Step](https://mlsanity.com/paper/2408.07884/) (2024)
- [Privacy Meets Explainability: A Comprehensive Impact Benchmark](https://mlsanity.com/paper/2211.04110/) (2022)
- [Detecting Gender Bias in Transformer-based Models: A Case Study on BERT](https://mlsanity.com/paper/2110.15733/) (2021)
- [ChatGPT-Like Large-Scale Foundation Models for Prognostics and Health Management: A Survey and Roadmaps](https://mlsanity.com/paper/2305.06472/) (2023)
- [Damage detection using in-domain and cross-domain transfer learning](https://mlsanity.com/paper/2102.03858/) (2021)
- [A Survey of Neural Trees](https://mlsanity.com/paper/2209.03415/) (2022)
- [Interpretability with full complexity by constraining feature information](https://mlsanity.com/paper/2211.17264/) (2022)
- [Unveiling the Unseen: A Comprehensive Survey on Explainable Anomaly Detection in Images and Videos](https://mlsanity.com/paper/2302.06670/) (2023)

## Abstract (arXiv)

Deep learning as represented by the artificial deep neural networks (DNNs) has achieved great success in many important areas that deal with text, images, videos, graphs, and so on. However, the black-box nature of DNNs has become one of the primary obstacles for their wide acceptance in mission-critical applications such as medical diagnosis and therapy. Due to the huge potential of deep learning, interpreting neural networks has recently attracted much research attention. In this paper, based on our comprehensive taxonomy, we systematically review recent studies in understanding the mechanism of neural networks, describe applications of interpretability especially in medicine, and discuss future directions of interpretability research, such as in relation to fuzzy logic and brain science.