# Robust Explainability: A Tutorial on Gradient-Based Attribution Methods for Deep Neural Networks

arXiv: https://arxiv.org/abs/2107.11400 · HTML: https://mlsanity.com/paper/2107.11400/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 40 references that are in the map.
- First posted on arXiv 23 Jul 2021.

## Builds on (most cited first, 30 of 32)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [Robustness May Be at Odds with Accuracy](https://mlsanity.com/paper/1805.12152/) (2018)
- [A Benchmark for Interpretability Methods in Deep Neural Networks](https://mlsanity.com/paper/1806.10758/) (2018)
- [Interpretation of Neural Networks is Fragile](https://mlsanity.com/paper/1710.10547/) (2017)
- [Evaluating the visualization of what a Deep Neural Network has learned](https://mlsanity.com/paper/1509.06321/) (2015)
- [European Union regulations on algorithmic decision-making and a "right to explanation"](https://mlsanity.com/paper/1606.08813/) (2016)
- [The (Un)reliability of saliency methods](https://mlsanity.com/paper/1711.00867/) (2017)
- [On the Robustness of Interpretability Methods](https://mlsanity.com/paper/1806.08049/) (2018)
- [On the (In)fidelity and Sensitivity for Explanations](https://mlsanity.com/paper/1901.09392/) (2019)
- [Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients](https://mlsanity.com/paper/1711.09404/) (2017)
- [Explanations can be manipulated and geometry is to blame](https://mlsanity.com/paper/1906.07983/) (2019)
- [Full-Gradient Representation for Neural Network Visualization](https://mlsanity.com/paper/1905.00780/) (2019)
- [Sanity Checks for Saliency Metrics](https://mlsanity.com/paper/1912.01451/) (2019)
- [On the Connection Between Adversarial Robustness and Saliency Map Interpretability](https://mlsanity.com/paper/1905.04172/) (2019)
- [What Does Explainable AI Really Mean? A New Conceptualization of Perspectives](https://mlsanity.com/paper/1710.00794/) (2017)
- [A Theoretical Explanation for Perplexing Behaviors of Backpropagation-based Visualizations](https://mlsanity.com/paper/1805.07039/) (2018)
- [Are Visual Explanations Useful? A Case Study in Model-in-the-Loop Prediction](https://mlsanity.com/paper/2007.12248/) (2020)
- [EXplainable Neural-Symbolic Learning (X-NeSyL) methodology to fuse deep learning representations with expert knowledge graphs: the MonuMAI cultural heritage use case](https://mlsanity.com/paper/2104.11914/) (2021)
- [Contrastive Explanations in Neural Networks](https://mlsanity.com/paper/2008.00178/) (2020)
- [Bridging Adversarial Robustness and Gradient Interpretability](https://mlsanity.com/paper/1903.11626/) (2019)
- [Distorted Representation Space Characterization Through Backpropagated Gradients](https://mlsanity.com/paper/1908.09998/) (2019)

## Built on by (most cited first, 30 of 48)

- [Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks](https://mlsanity.com/paper/2207.13243/) (2022)
- [Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions](https://mlsanity.com/paper/2310.19775/) (2023)
- [Tracr: Compiled Transformers as a Laboratory for Interpretability](https://mlsanity.com/paper/2301.05062/) (2023)
- [Trustworthy Graph Neural Networks: Aspects, Methods and Trends](https://mlsanity.com/paper/2205.07424/) (2022)
- [Transformers in Time-series Analysis: A Tutorial](https://mlsanity.com/paper/2205.01138/) (2022)
- [Encoding Time-Series Explanations through Self-Supervised Model Behavior Consistency](https://mlsanity.com/paper/2306.02109/) (2023)
- [Red Teaming Deep Neural Networks with Feature Synthesis Tools](https://mlsanity.com/paper/2302.10894/) (2023)
- [Multimodal Data Integration for Oncology in the Era of Deep Neural Networks: A Review](https://mlsanity.com/paper/2303.06471/) (2023)
- [Enhancing Visual Question Answering through Ranking-Based Hybrid Training and Multimodal Fusion](https://mlsanity.com/paper/2408.07303/) (2024)
- [Evaluating the Robustness of Interpretability Methods through Explanation Invariance and Equivariance](https://mlsanity.com/paper/2304.06715/) (2023)
- [Explaining Full-disk Deep Learning Model for Solar Flare Prediction using Attribution Methods](https://mlsanity.com/paper/2307.15878/) (2023)
- [Privacy Meets Explainability: A Comprehensive Impact Benchmark](https://mlsanity.com/paper/2211.04110/) (2022)
- [Time to Focus: A Comprehensive Benchmark Using Time Series Attribution Methods](https://mlsanity.com/paper/2202.03759/) (2022)
- [Gradient-based Uncertainty Attribution for Explainable Bayesian Deep Learning](https://mlsanity.com/paper/2304.04824/) (2023)
- [SoK: Modeling Explainability in Security Analytics for Interpretability, Trustworthiness, and Usability](https://mlsanity.com/paper/2210.17376/) (2022)
- [Explainable Deep Learning-based Solar Flare Prediction with post hoc Attention for Operational Forecasting](https://mlsanity.com/paper/2308.02682/) (2023)
- [Exploring Deep Learning for Full-disk Solar Flare Prediction with Empirical Insights from Guided Grad-CAM Explanations](https://mlsanity.com/paper/2308.15712/) (2023)
- [Feature Attribution with Necessity and Sufficiency via Dual-stage Perturbation Test for Causal Explanation](https://mlsanity.com/paper/2402.08845/) (2024)
- [EvalAttAI: A Holistic Approach to Evaluating Attribution Maps in Robust and Non-Robust Models](https://mlsanity.com/paper/2303.08866/) (2023)
- [Towards Explainable AI for Channel Estimation in Wireless Communications](https://mlsanity.com/paper/2307.00952/) (2023)
- [Noiser: Bounded Input Perturbations for Attributing Large Language Models](https://mlsanity.com/paper/2504.02911/) (2025)
- [Trainable Noise Model as an XAI evaluation method: application on Sobol for remote sensing image segmentation](https://mlsanity.com/paper/2310.01828/) (2023)
- [Extending CAM-based XAI methods for Remote Sensing Imagery Segmentation](https://mlsanity.com/paper/2310.01837/) (2023)
- [Towards Explaining Hypercomplex Neural Networks](https://mlsanity.com/paper/2403.17929/) (2024)
- [A Gradient Mapping Guided Explainable Deep Neural Network for Extracapsular Extension Identification in 3D Head and Neck Cancer Computed Tomography Images](https://mlsanity.com/paper/2201.00895/) (2022)
- [Fixing confirmation bias in feature attribution methods via semantic match](https://mlsanity.com/paper/2307.00897/) (2023)
- [SPADE: Sparsity-Guided Debugging for Deep Neural Networks](https://mlsanity.com/paper/2310.04519/) (2023)
- [Explainable AI for Enhancing Efficiency of DL-based Channel Estimation](https://mlsanity.com/paper/2407.07009/) (2024)
- [Provable Robust Saliency-based Explanations](https://mlsanity.com/paper/2212.14106/) (2022)
- [The Topos of Transformer Networks](https://mlsanity.com/paper/2403.18415/) (2024)

## Abstract (arXiv)

With the rise of deep neural networks, the challenge of explaining the predictions of these networks has become increasingly recognized. While many methods for explaining the decisions of deep neural networks exist, there is currently no consensus on how to evaluate them. On the other hand, robustness is a popular topic for deep learning research; however, it is hardly talked about in explainability until very recently. In this tutorial paper, we start by presenting gradient-based interpretability methods. These techniques use gradient signals to assign the burden of the decision on the input features. Later, we discuss how gradient-based methods can be evaluated for their robustness and the role that adversarial robustness plays in having meaningful explanations. We also discuss the limitations of gradient-based methods. Finally, we present the best practices and attributes that should be examined before choosing an explainability method. We conclude with the future directions for research in the area at the convergence of robustness and explainability.