# Testing the robustness of attribution methods for convolutional neural networks in MRI-based Alzheimer's disease classification

arXiv: https://arxiv.org/abs/1909.08856 · HTML: https://mlsanity.com/paper/1909.08856/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Sep 2019.

## Built on by (most cited first, 25 of 25)

- [Explainable artificial intelligence (XAI) in deep learning-based medical image analysis](https://mlsanity.com/paper/2107.10912/) (2021)
- [Transparency of Deep Neural Networks for Medical Image Analysis: A Review of Interpretability Methods](https://mlsanity.com/paper/2111.02398/) (2021)
- [Explainable AI: current status and future directions](https://mlsanity.com/paper/2107.07045/) (2021)
- [Explainable Deep Learning Methods in Medical Image Classification: A Survey](https://mlsanity.com/paper/2205.04766/) (2022)
- [Diffusion Models for Counterfactual Generation and Anomaly Detection in Brain Images](https://mlsanity.com/paper/2308.02062/) (2023)
- [Doubly Right Object Recognition: A Why Prompt for Visual Rationales](https://mlsanity.com/paper/2212.06202/) (2022)
- [ICAM-reg: Interpretable Classification and Regression with Feature Attribution for Mapping Neurological Phenotypes in Individual Scans](https://mlsanity.com/paper/2103.02561/) (2021)
- [A comprehensive study on fidelity metrics for XAI](https://mlsanity.com/paper/2401.10640/) (2024)
- [Explainable Deep Learning in Healthcare: A Methodological Survey from an Attribution View](https://mlsanity.com/paper/2112.02625/) (2021)
- [Assessing Fidelity in XAI post-hoc techniques: A Comparative Study with Ground Truth Explanations Datasets](https://mlsanity.com/paper/2311.01961/) (2023)
- [FUTURE-AI: Guiding Principles and Consensus Recommendations for Trustworthy Artificial Intelligence in Medical Imaging](https://mlsanity.com/paper/2109.09658/) (2021)
- [ACAT: Adversarial Counterfactual Attention for Classification and Detection in Medical Imaging](https://mlsanity.com/paper/2303.15421/) (2023)
- [Counterfactual Explanation of Brain Activity Classifiers using Image-to-Image Transfer by Generative Adversarial Network](https://mlsanity.com/paper/2110.14927/) (2021)
- [Information Bottleneck Attribution for Visual Explanations of Diagnosis and Prognosis](https://mlsanity.com/paper/2104.02869/) (2021)
- [Interpretability of Machine Learning Methods Applied to Neuroimaging](https://mlsanity.com/paper/2204.07005/) (2022)
- [Promises and pitfalls of deep neural networks in neuroimaging-based psychiatric research](https://mlsanity.com/paper/2301.08525/) (2023)
- [Evaluating saliency methods on artificial data with different background types](https://mlsanity.com/paper/2112.04882/) (2021)
- [Applications of interpretable deep learning in neuroimaging: a comprehensive review](https://mlsanity.com/paper/2406.17792/) (2024)
- [SurfGNN: A robust surface-based prediction model with interpretability for coactivation maps of spatial and cortical features](https://mlsanity.com/paper/2411.05825/) (2024)
- [CheXseg: Combining Expert Annotations with DNN-generated Saliency Maps for X-ray Segmentation](https://mlsanity.com/paper/2102.10484/) (2021)
- [Explaining the Black-box Smoothly- A Counterfactual Approach](https://mlsanity.com/paper/2101.04230/) (2021)
- [Disentangled representations: towards interpretation of sex determination from hip bone](https://mlsanity.com/paper/2112.09414/) (2021)
- [Multi-modal volumetric concept activation to explain detection and classification of metastatic prostate cancer on PSMA-PET/CT](https://mlsanity.com/paper/2208.02555/) (2022)
- [A Sneak Attack on Segmentation of Medical Images Using Deep Neural Network Classifiers](https://mlsanity.com/paper/2201.02771/) (2022)
- [Towards a Novel Measure of User Trust in XAI Systems](https://mlsanity.com/paper/2405.05766/) (2024)

## Abstract (arXiv)

Attribution methods are an easy to use tool for investigating and validating machine learning models. Multiple methods have been suggested in the literature and it is not yet clear which method is most suitable for a given task. In this study, we tested the robustness of four attribution methods, namely gradient*input, guided backpropagation, layer-wise relevance propagation and occlusion, for the task of Alzheimer's disease classification. We have repeatedly trained a convolutional neural network (CNN) with identical training settings in order to separate structural MRI data of patients with Alzheimer's disease and healthy controls. Afterwards, we produced attribution maps for each subject in the test data and quantitatively compared them across models and attribution methods. We show that visual comparison is not sufficient and that some widely used attribution methods produce highly inconsistent outcomes.