mlsanityData to 7 Oct 2026

Robust Explainability: A Tutorial on Gradient-Based Attribution Methods for Deep Neural Networks

arXiv 2107.11400 ↗First posted 23 Jul 2021cs.LG cs.AI cs.CVcited by 48

Not on the map. Why?

Abstract (arXiv)

With the rise of deep neural networks, the challenge of explaining the predictions of these networks has become increasingly recognized. While many methods for explaining the decisions of deep neural networks exist, there is currently no consensus on how to evaluate them. On the other hand, robustness is a popular topic for deep learning research; however, it is hardly talked about in explainability until very recently. In this tutorial paper, we start by presenting gradient-based interpretability methods. These techniques use gradient signals to assign the burden of the decision on the input features. Later, we discuss how gradient-based methods can be evaluated for their robustness and the role that adversarial robustness plays in having meaningful explanations. We also discuss the limitations of gradient-based methods. Finally, we present the best practices and attributes that should be examined before choosing an explainability method. We conclude with the future directions for research in the area at the convergence of robustness and explainability.

Key facts. Cited by 48 papers in the map (data through 7 Oct 2026). Builds on 32 of its 40 references that are in the map. First posted on arXiv 23 Jul 2021.

Builds on

32 of 40 references.

References · 32

  1. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  4. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  5. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  6. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,543
  7. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  8. 2018Sanity Checks for Saliency Mapscited by 1,149
  9. 2017SmoothGrad: removing noise by adding noisecited by 1,097
  10. 2014Striving for Simplicity: The All Convolutional Netcited by 983
  11. 2018Robustness May Be at Odds with Accuracycited by 873
  12. 2018A Benchmark for Interpretability Methods in Deep Neural Networkscited by 467
20 more
  1. 2017Interpretation of Neural Networks is Fragilecited by 443
  2. 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 416
  3. 2016European Union regulations on algorithmic decision-making and a "right to explanation"cited by 343
  4. 2017The (Un)reliability of saliency methodscited by 331
  5. 2018On the Robustness of Interpretability Methodscited by 310
  6. 2019On the (In)fidelity and Sensitivity for Explanationscited by 309
  7. 2017Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradientscited by 236
  8. 2019Explanations can be manipulated and geometry is to blamecited by 215
  9. 2019Full-Gradient Representation for Neural Network Visualizationcited by 189
  10. 2019Sanity Checks for Saliency Metricscited by 119
  11. 2019On the Connection Between Adversarial Robustness and Saliency Map Interpretabilitycited by 80
  12. 2017What Does Explainable AI Really Mean? A New Conceptualization of Perspectivescited by 72
  13. 2018A Theoretical Explanation for Perplexing Behaviors of Backpropagation-based Visualizationscited by 59
  14. 2020Are Visual Explanations Useful? A Case Study in Model-in-the-Loop Predictioncited by 36
  15. 2021EXplainable Neural-Symbolic Learning (X-NeSyL) methodology to fuse deep learning representations with expert knowledge graphs: the MonuMAI cultural heritage use casecited by 30
  16. 2020Contrastive Explanations in Neural Networkscited by 26
  17. 2019Bridging Adversarial Robustness and Gradient Interpretabilitycited by 22
  18. 2019Distorted Representation Space Characterization Through Backpropagated Gradientscited by 12
  19. 2021Building Reliable Explanations of Unreliable Neural Networks: Locally Smoothing Perspective of Model Interpretationcited by 12
  20. 2021Contrastive Reasoning in Neural Networkscited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 10; 2023: 18; 2024: 11; 2025: 8; 2026: 1.

48 papers.

Sort:

By area:

  1. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
  2. 2023Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directionscited by 123
  3. 2023Tracr: Compiled Transformers as a Laboratory for Interpretabilitycited by 99
  4. 2022Trustworthy Graph Neural Networks: Aspects, Methods and Trendscited by 88
  5. 2022Transformers in Time-series Analysis: A Tutorialcited by 45
  6. 2023Encoding Time-Series Explanations through Self-Supervised Model Behavior Consistencycited by 42
  7. 2023Red Teaming Deep Neural Networks with Feature Synthesis Toolscited by 21
  8. 2023Multimodal Data Integration for Oncology in the Era of Deep Neural Networks: A Reviewcited by 20
  9. 2024Enhancing Visual Question Answering through Ranking-Based Hybrid Training and Multimodal Fusioncited by 16
  10. 2023Evaluating the Robustness of Interpretability Methods through Explanation Invariance and Equivariancecited by 13
  11. 2023Explaining Full-disk Deep Learning Model for Solar Flare Prediction using Attribution Methodscited by 12
  12. 2022Privacy Meets Explainability: A Comprehensive Impact Benchmarkcited by 9
36 more
  1. 2022Time to Focus: A Comprehensive Benchmark Using Time Series Attribution Methodscited by 8
  2. 2023Gradient-based Uncertainty Attribution for Explainable Bayesian Deep Learningcited by 8
  3. 2022SoK: Modeling Explainability in Security Analytics for Interpretability, Trustworthiness, and Usabilitycited by 7
  4. 2023Explainable Deep Learning-based Solar Flare Prediction with post hoc Attention for Operational Forecastingcited by 7
  5. 2023Exploring Deep Learning for Full-disk Solar Flare Prediction with Empirical Insights from Guided Grad-CAM Explanationscited by 6
  6. 2024Feature Attribution with Necessity and Sufficiency via Dual-stage Perturbation Test for Causal Explanationcited by 6
  7. 2023EvalAttAI: A Holistic Approach to Evaluating Attribution Maps in Robust and Non-Robust Modelscited by 5
  8. 2023Towards Explainable AI for Channel Estimation in Wireless Communicationscited by 5
  9. 2025Noiser: Bounded Input Perturbations for Attributing Large Language Modelscited by 4
  10. 2023Trainable Noise Model as an XAI evaluation method: application on Sobol for remote sensing image segmentationcited by 3
  11. 2023Extending CAM-based XAI methods for Remote Sensing Imagery Segmentationcited by 3
  12. 2024Towards Explaining Hypercomplex Neural Networkscited by 3
  13. 2022A Gradient Mapping Guided Explainable Deep Neural Network for Extracapsular Extension Identification in 3D Head and Neck Cancer Computed Tomography Imagescited by 2
  14. 2023Fixing confirmation bias in feature attribution methods via semantic matchcited by 2
  15. 2023SPADE: Sparsity-Guided Debugging for Deep Neural Networkscited by 2
  16. 2024Explainable AI for Enhancing Efficiency of DL-based Channel Estimationcited by 2
  17. 2022Provable Robust Saliency-based Explanationscited by 1
  18. 2024The Topos of Transformer Networkscited by 1
  19. 2024FovEx: Human-Inspired Explanations for Vision Transformers and Convolutional Neural Networkscited by 1
  20. 2024Explainability in AI Based Applications: A Framework for Comparing Different Techniquescited by 1
  21. 2022Parameter Averaging for Feature Rankingcited by 0
  22. 2022Explainable Deep Learning to Profile Mitochondrial Disease Using High Dimensional Protein Expression Datacited by 0
  23. 2023Targeted Background Removal Creates Interpretable Feature Visualizationscited by 0
  24. 2023Robust Ranking Explanationscited by 0
  25. 2024Model-Agnostic Interpretation Framework in Machine Learning: A Comparative Study in NBA Sportscited by 0
  26. 2024Stability of Explainable Recommendationcited by 0
  27. 2024Selecting Interpretability Techniques for Healthcare Machine Learning modelscited by 0
  28. 2024On Diversity in Discriminative Neural Networkscited by 0
  29. 2025Algebraic Adversarial Attacks on Explainability Modelscited by 0
  30. 2025FREQuency ATTribution: benchmarking frequency-based occlusion for time series datacited by 0
  31. 2025Explainable AI in Genomics: Transcription Factor Binding Site Prediction with Mixture of Expertscited by 0
  32. 2025Chat-CBM: Towards Interactive Concept Bottleneck Models with Frozen Large Language Modelscited by 0
  33. 2025EVO-LRP: Evolutionary Optimization of LRP for Interpretable Model Explanationscited by 0
  34. 2025Identifying Autism-Related Neurobiomarkers Using Hybrid Deep Learning Modelscited by 0
  35. 2025Personalized Treatment Outcome Prediction from Scarce Data via Dual-Channel Knowledge Distillation and Adaptive Fusioncited by 0
  36. 2026FADEx: Feature Attribution and Distortion-based Explanation of Dimensionality Reductioncited by 0