# Towards Better Understanding Attribution Methods

arXiv: https://arxiv.org/abs/2205.10435 · HTML: https://mlsanity.com/paper/2205.10435/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 36 references that are in the map.
- First posted on arXiv 20 May 2022.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [Interpretable Explanations of Black Boxes by Meaningful Perturbation](https://mlsanity.com/paper/1704.03296/) (2017)
- [Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks](https://mlsanity.com/paper/1910.01279/) (2019)
- [Understanding the Effective Receptive Field in Deep Convolutional Neural Networks](https://mlsanity.com/paper/1701.04128/) (2017)
- [Evaluating the visualization of what a Deep Neural Network has learned](https://mlsanity.com/paper/1509.06321/) (2015)
- [Top-down Neural Attention by Excitation Backprop](https://mlsanity.com/paper/1608.00507/) (2016)
- [Real Time Image Saliency for Black Box Classifiers](https://mlsanity.com/paper/1705.07857/) (2017)
- [Full-Gradient Representation for Neural Network Visualization](https://mlsanity.com/paper/1905.00780/) (2019)
- [Restricting the Flow: Information Bottlenecks for Attribution](https://mlsanity.com/paper/2001.00396/) (2020)
- [There and Back Again: Revisiting Backpropagation Saliency Methods](https://mlsanity.com/paper/2004.02866/) (2020)
- [Convolutional Dynamic Alignment Networks for Interpretable Classifications](https://mlsanity.com/paper/2104.00032/) (2021)
- [Do Input Gradients Highlight Discriminative Features?](https://mlsanity.com/paper/2102.12781/) (2021)
- [Not Using the Car to See the Sidewalk: Quantifying and Controlling the Effects of Context in Classification and Segmentation](https://mlsanity.com/paper/1812.06707/) (2018)
- [Role of Spatial Context in Adversarial Robustness for Object Detection](https://mlsanity.com/paper/1910.00068/) (2019)

## Built on by (most cited first, 28 of 28)

- [A Holistic Approach to Unifying Automatic Concept Extraction and Concept Importance Estimation](https://mlsanity.com/paper/2306.07304/) (2023)
- [FunnyBirds: A Synthetic Vision Dataset for a Part-Based Analysis of Explainable AI Methods](https://mlsanity.com/paper/2308.06248/) (2023)
- [Don't trust your eyes: on the (un)reliability of feature visualizations](https://mlsanity.com/paper/2306.04719/) (2023)
- [B-cos Alignment for Inherently Interpretable CNNs and Vision Transformers](https://mlsanity.com/paper/2306.10898/) (2023)
- [Unlocking Feature Visualization for Deeper Networks with MAgnitude Constrained Optimization](https://mlsanity.com/paper/2306.06805/) (2023)
- [Opti-CAM: Optimizing saliency maps for interpretability](https://mlsanity.com/paper/2301.07002/) (2023)
- [SpArX: Sparse Argumentative Explanations for Neural Networks [Technical Report]](https://mlsanity.com/paper/2301.09559/) (2023)
- [Studying How to Efficiently and Effectively Guide Models with Explanations](https://mlsanity.com/paper/2303.11932/) (2023)
- [B-cosification: Transforming Deep Neural Networks to be Inherently Interpretable](https://mlsanity.com/paper/2411.00715/) (2024)
- [Classification Metrics for Image Explanations: Towards Building Reliable XAI-Evaluations](https://mlsanity.com/paper/2406.05068/) (2024)
- [A Survey of Explainable AI in Deep Visual Modeling: Methods and Metrics](https://mlsanity.com/paper/2301.13445/) (2023)
- [Neural Insights for Digital Marketing Content Design](https://mlsanity.com/paper/2302.01416/) (2023)
- [Better Understanding Differences in Attribution Methods via Systematic Evaluations](https://mlsanity.com/paper/2303.11884/) (2023)
- [What Sketch Explainability Really Means for Downstream Tasks](https://mlsanity.com/paper/2403.09480/) (2024)
- [Red-Teaming Segment Anything Model](https://mlsanity.com/paper/2404.02067/) (2024)
- [Saliency strikes back: How filtering out high frequencies improves white-box explanations](https://mlsanity.com/paper/2307.09591/) (2023)
- [Parallel Backpropagation for Shared-Feature Visualization](https://mlsanity.com/paper/2405.09827/) (2024)
- [Analyzing Effects of Mixed Sample Data Augmentation on Model Interpretability](https://mlsanity.com/paper/2303.14608/) (2023)
- [A Backdoor-based Explainable AI Benchmark for High Fidelity Evaluation of Attributions](https://mlsanity.com/paper/2405.02344/) (2024)
- [Attribution Explanations for Deep Neural Networks: A Theoretical Perspective](https://mlsanity.com/paper/2508.07636/) (2025)
- [Illuminating Salient Contributions in Neuron Activation with Attribution Equilibrium](https://mlsanity.com/paper/2205.11109/) (2022)
- [Certified ℓ2 Attribution Robustness via Uniformly Smoothed Attributions](https://mlsanity.com/paper/2405.06361/) (2024)
- [Hidden in Plain Sight -- Class Competition Focuses Attribution Maps](https://mlsanity.com/paper/2503.07346/) (2025)
- [Exploring SAIG Methods for an Objective Evaluation of XAI](https://mlsanity.com/paper/2602.08715/) (2026)
- [Evaluating Feature Attribution Methods for Electrocardiogram](https://mlsanity.com/paper/2211.12702/) (2022)
- [Smooth Deep Saliency](https://mlsanity.com/paper/2404.02282/) (2024)
- [Toward Faithful Segmentation Attribution via Benchmarking and Dual-Evidence Fusion](https://mlsanity.com/paper/2603.22624/) (2026)
- [HiLRP: Toward One Trustworthy Explanation for Vision Transformer: Conservation-Valid Attribution via Attention Primitives](https://mlsanity.com/paper/2609.01282/) (2026)

## Abstract (arXiv)

Deep neural networks are very successful on many vision tasks, but hard to interpret due to their black box nature. To overcome this, various post-hoc attribution methods have been proposed to identify image regions most influential to the models' decisions. Evaluating such methods is challenging since no ground truth attributions exist. We thus propose three novel evaluation schemes to more reliably measure the faithfulness of those methods, to make comparisons between them more fair, and to make visual inspection more systematic. To address faithfulness, we propose a novel evaluation setting (DiFull) in which we carefully control which parts of the input can influence the output in order to distinguish possible from impossible attributions. To address fairness, we note that different methods are applied at different layers, which skews any comparison, and so evaluate all methods on the same layers (ML-Att) and discuss how this impacts their performance on quantitative metrics. For more systematic visualizations, we propose a scheme (AggAtt) to qualitatively evaluate the methods on complete datasets. We use these evaluation schemes to study strengths and shortcomings of some widely used attribution methods. Finally, we propose a post-processing smoothing step that significantly improves the performance of some attribution methods, and discuss its applicability.