mlsanityData to 7 Oct 2026

Evaluating Feature Attribution Methods in the Image Domain

arXiv 2202.12270 ↗First posted 22 Feb 2022cs.CV cs.LGcited by 19

Not on the map. Why?

Abstract (arXiv)

Feature attribution maps are a popular approach to highlight the most important pixels in an image for a given prediction of a model. Despite a recent growth in popularity and available methods, little attention is given to the objective evaluation of such attribution maps. Building on previous work in this domain, we investigate existing metrics and propose new variants of metrics for the evaluation of attribution maps. We confirm a recent finding that different attribution metrics seem to measure different underlying concepts of attribution maps, and extend this finding to a larger selection of attribution metrics. We also find that metric results on one dataset do not necessarily generalize to other datasets, and methods with desirable theoretical properties such as DeepSHAP do not necessarily outperform computationally cheaper alternatives. Based on these findings, we propose a general benchmarking approach to identify the ideal feature attribution method for a given use case. Implementations of attribution metrics and our experiments are available online.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 46 of its 89 references that are in the map. First posted on arXiv 22 Feb 2022.

Builds on

46 of 89 references.

References · 46

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  6. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  7. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  8. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  9. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  10. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  11. 2015Continuous control with deep reinforcement learningcited by 3,279
  12. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
34 more
  1. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  2. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  3. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  4. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  5. 2018Sanity Checks for Saliency Mapscited by 1,149
  6. 2017SmoothGrad: removing noise by adding noisecited by 1,097
  7. 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,086
  8. 2014Striving for Simplicity: The All Convolutional Netcited by 983
  9. 2017Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networkscited by 936
  10. 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 825
  11. 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 611
  12. 2019Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networkscited by 575
  13. 2020Captum: A unified and generic model interpretability library for PyTorchcited by 542
  14. 2014OpenML: networked science in machine learningcited by 523
  15. 2018A Benchmark for Interpretability Methods in Deep Neural Networkscited by 467
  16. 2017Interpretation of Neural Networks is Fragilecited by 443
  17. 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 416
  18. 2019The many Shapley values for model explanationcited by 341
  19. 2019On the (In)fidelity and Sensitivity for Explanationscited by 309
  20. 2020Multi-Objective Counterfactual Explanationscited by 157
  21. 2017The Shattered Gradients Problem: If resnets are the answer, then what is the question?cited by 156
  22. 2021Explainable artificial intelligence (XAI) in deep learning-based medical image analysiscited by 147
  23. 2020True to the Model or True to the Data?cited by 123
  24. 2019Sanity Checks for Saliency Metricscited by 119
  25. 2019XRAI: Better Attributions Through Regionscited by 116
  26. 2019Benchmarking Attribution Methods with Relative Feature Importancecited by 89
  27. 2019Quantifying Interpretability and Trust in Machine Learning Systemscited by 49
  28. 2020IROF: a low resource evaluation metric for explanation methodscited by 45
  29. 2018Improving the Interpretability of Deep Neural Networks with Knowledge Distillationcited by 37
  30. 2019Do Explanations Reflect Decisions? A Machine-centric Strategy to Quantify the Performance of Explainability Algorithmscited by 32
  31. 2020Assessing the (Un)Trustworthiness of Saliency Maps for Localizing Abnormalities in Medical Imagingcited by 32
  32. 2021Revisiting Sanity Checks for Saliency Mapscited by 20
  33. 2020Assessing the validity of saliency maps for abnormality localization in medical imagingcited by 9
  34. 2020Towards Trainable Saliency Maps in Medical Imagingcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 6; 2024: 5; 2025: 3; 2026: 3.

19 papers.

Sort:

  1. 2023The Meta-Evaluation Problem in Explainable AI: Identifying Reliable Estimators with MetaQuantuscited by 37
  2. 2024Decoupling Pixel Flipping and Occlusion Strategy for Consistent XAI Benchmarkscited by 28
  3. 2023Finding the right XAI method -- A Guide for the Evaluation and Ranking of Explainable AI Methods in Climate Sciencecited by 25
  4. 2022How explainable are adversarially-robust CNNs?cited by 9
  5. 2023XAI-TRIS: Non-linear image benchmarks to quantify false positive post-hoc attribution of feature importancecited by 7
  6. 2024Benchmarking the Attribution Quality of Vision Modelscited by 7
  7. 2025Tokenize Image Patches: Global Context Fusion for Effective Haze Removal in Large Imagescited by 7
  8. 2024SPES: Spectrogram Perturbation for Explainable Speech-to-Text Generationcited by 6
  9. 2024Toward Understanding the Disagreement Problem in Neural Network Feature Attributioncited by 5
  10. 2025The Unheard Alternative: Contrastive Explanations for Speech-to-Text Modelscited by 3
  11. 2023Shapley Sets: Feature Attribution via Recursive Function Decompositioncited by 2
  12. 2023Explaining and visualizing black-box models through counterfactual pathscited by 2
7 more
  1. 2025Attribution Explanations for Deep Neural Networks: A Theoretical Perspectivecited by 2
  2. 2024On the Black-box Explainability of Object Detection Models for Safe and Trustworthy Industrial Applicationscited by 1
  3. 2026The effect of whitening on explanation performancecited by 1
  4. 2022Evaluating Feature Attribution Methods for Electrocardiogramcited by 0
  5. 2023Explanation-based Training with Differentiable Insertion/Deletion Metric-aware Regularizerscited by 0
  6. 2026GRALIS: Fusing Coalition and Gradient Attribution with Closed-Form Conservation Error and Finite-Sample Guaranteescited by 0
  7. 2026XtrAIn: Training-Guided Occlusion for Feature Attributioncited by 0