# Explainable Deep Learning Methods in Medical Image Classification: A Survey

arXiv: https://arxiv.org/abs/2205.04766 · HTML: https://mlsanity.com/paper/2205.04766/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 85 of its 182 references that are in the map.
- First posted on arXiv 10 May 2022.

## Builds on (most cited first, 30 of 85)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [Deep Clustering for Unsupervised Learning of Visual Features](https://mlsanity.com/paper/1807.05520/) (2018)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases](https://mlsanity.com/paper/1705.02315/) (2017)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [A Benchmark for Interpretability Methods in Deep Neural Networks](https://mlsanity.com/paper/1806.10758/) (2018)
- [Interpretation of Neural Networks is Fragile](https://mlsanity.com/paper/1710.10547/) (2017)
- [Evaluating the visualization of what a Deep Neural Network has learned](https://mlsanity.com/paper/1509.06321/) (2015)

## Built on by (most cited first, 30 of 32)

- [Self-eXplainable AI for Medical Image Analysis: A Survey and New Outlooks](https://mlsanity.com/paper/2410.02331/) (2024)
- [Probabilistic Stability Guarantees for Feature Attributions](https://mlsanity.com/paper/2504.13787/) (2025)
- [InfoDisent: Explainability of Image Classification Models by Information Disentanglement](https://mlsanity.com/paper/2409.10329/) (2024)
- [DiffExplainer: Unveiling Black Box Models Via Counterfactual Generation](https://mlsanity.com/paper/2406.15182/) (2024)
- [DWARF: Disease-weighted network for attention map refinement](https://mlsanity.com/paper/2406.17032/) (2024)
- [Navigating Distribution Shifts in Medical Image Analysis: A Survey](https://mlsanity.com/paper/2411.05824/) (2024)
- [Cross- and Intra-image Prototypical Learning for Multi-label Disease Diagnosis and Interpretation](https://mlsanity.com/paper/2411.04607/) (2024)
- [X-Ray-CoT: Interpretable Chest X-ray Diagnosis with Vision-Language Models via Chain-of-Thought Reasoning](https://mlsanity.com/paper/2508.12455/) (2025)
- [This looks like what? Challenges and Future Research Directions for Part-Prototype Models](https://mlsanity.com/paper/2502.09340/) (2025)
- [A Systematic Literature Review on Explainability for Machine/Deep Learning-based Software Engineering Research](https://mlsanity.com/paper/2401.14617/) (2024)
- [Explainable Interface for Human-Autonomy Teaming: A Survey](https://mlsanity.com/paper/2405.02583/) (2024)
- [Comprehensive Review of Reinforcement Learning for Medical Ultrasound Imaging](https://mlsanity.com/paper/2503.16543/) (2025)
- [Can Physician Judgment Enhance Model Trustworthiness? A Case Study on Predicting Pathological Lymph Nodes in Rectal Cancer](https://mlsanity.com/paper/2312.09529/) (2023)
- [MAProtoNet: A Multi-scale Attentive Interpretable Prototypical Part Network for 3D Magnetic Resonance Imaging Brain Tumor Classification](https://mlsanity.com/paper/2404.08917/) (2024)
- [Unsupervised contrastive analysis for anomaly detection in brain MRIs via conditional diffusion models](https://mlsanity.com/paper/2406.00772/) (2024)
- [Lost in Context: The Influence of Context on Feature Attribution Methods for Object Recognition](https://mlsanity.com/paper/2411.02833/) (2024)
- [Explainable AI the Latest Advancements and New Trends](https://mlsanity.com/paper/2505.07005/) (2025)
- [Explainable Artificial Intelligence in Biomedical Image Analysis: A Comprehensive Survey](https://mlsanity.com/paper/2507.07148/) (2025)
- [T-DuMpRa: Teacher-guided Dual-path Multi-prototype Retrieval Augmented framework for fine-grained medical image classification](https://mlsanity.com/paper/2604.17360/) (2026)
- [Interaction as Explanation: A User Interaction-based Method for Explaining Image Classification Models](https://mlsanity.com/paper/2404.09828/) (2024)
- [Artificial Intelligence in Bone Metastasis Analysis: Current Advancements, Opportunities and Challenges](https://mlsanity.com/paper/2404.19598/) (2024)
- [Shape Modeling of Longitudinal Medical Images: From Diffeomorphic Metric Mapping to Deep Learning](https://mlsanity.com/paper/2503.21489/) (2025)
- [Secure Diagnostics: Adversarial Robustness Meets Clinical Interpretability](https://mlsanity.com/paper/2504.05483/) (2025)
- [VL-MedGuide: A Visual-Linguistic Large Model for Intelligent and Explainable Skin Disease Auxiliary Diagnosis](https://mlsanity.com/paper/2508.06624/) (2025)
- [Explainable Graph Neural Networks: Understanding Brain Connectivity and Biomarkers in Dementia](https://mlsanity.com/paper/2509.18568/) (2025)
- [MultiFair: Multimodal Balanced Fairness-Aware Medical Classification with Dual-Level Gradient Modulation](https://mlsanity.com/paper/2510.07328/) (2025)
- [ViConEx-Med: Visual Concept Explainability via Multi-Concept Token Transformer for Medical Image Analysis](https://mlsanity.com/paper/2510.10174/) (2025)
- [SkinGenBench: Generative Model and Preprocessing Effects for Synthetic Dermoscopic Augmentation in Melanoma Diagnosis](https://mlsanity.com/paper/2512.17585/) (2025)
- [DCG-Net: Dual Cross-Attention with Concept-Value Graph Reasoning for Interpretable Medical Diagnosis](https://mlsanity.com/paper/2603.20325/) (2026)
- [Secure-by-Disguise: A Systematic Evaluation of Image Disguising for Confidential Medical Image Modeling](https://mlsanity.com/paper/2607.08867/) (2026)

## Abstract (arXiv)

The remarkable success of deep learning has prompted interest in its application to medical imaging diagnosis. Even though state-of-the-art deep learning models have achieved human-level accuracy on the classification of different types of medical data, these models are hardly adopted in clinical workflows, mainly due to their lack of interpretability. The black-box-ness of deep learning models has raised the need for devising strategies to explain the decision process of these models, leading to the creation of the topic of eXplainable Artificial Intelligence (XAI). In this context, we provide a thorough survey of XAI applied to medical imaging diagnosis, including visual, textual, example-based and concept-based explanation methods. Moreover, this work reviews the existing medical imaging datasets and the existing metrics for evaluating the quality of the explanations. In addition, we include a performance comparison among a set of report generation-based methods. Finally, the major challenges in applying XAI to medical imaging and the future research directions on the topic are also discussed.