mlsanityData to 7 Oct 2026

Benchmarking the Attribution Quality of Vision Models

arXiv 2407.11910 ↗First posted 16 Jul 2024cs.CV cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Attribution maps are one of the most established tools to explain the functioning of computer vision models. They assign importance scores to input features, indicating how relevant each feature is for the prediction of a deep neural network. While much research has gone into proposing new attribution methods, their proper evaluation remains a difficult challenge. In this work, we propose a novel evaluation protocol that overcomes two fundamental limitations of the widely used incremental-deletion protocol, i.e., the out-of-domain issue and lacking inter-model comparisons. This allows us to evaluate 23 attribution methods and how different design choices of popular vision backbones affect their attribution quality. We find that intrinsically explainable models outperform standard models and that raw attribution values exhibit a higher attribution quality than what is known from previous work. Further, we show consistent changes in the attribution quality when varying the network design, indicating that some standard design choices promote attribution quality.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 58 of its 74 references that are in the map. First posted on arXiv 16 Jul 2024.

Builds on

58 of 74 references.

References · 58

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  6. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  7. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  8. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  9. 2016Wide Residual Networkscited by 3,057
  10. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  11. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  12. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
38 more
  1. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  2. 2017SmoothGrad: removing noise by adding noisecited by 1,097
  3. 2017Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networkscited by 936
  4. 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 825
  5. 2020Quantifying Attention Flow in Transformerscited by 759
  6. 2020Transformer Interpretability Beyond Attention Visualizationcited by 620
  7. 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 611
  8. 2020Captum: A unified and generic model interpretability library for PyTorchcited by 542
  9. 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
  10. 2018A Benchmark for Interpretability Methods in Deep Neural Networkscited by 467
  11. 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 416
  12. 2017Methods for Interpreting and Understanding Deep Neural Networkscited by 342
  13. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 299
  14. 2016Not Just a Black Box: Learning Important Features Through Propagating Activation Differencescited by 295
  15. 2017Right for the Right Reasons: Training Differentiable Models by Constraining their Explanationscited by 293
  16. 2019Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNetcited by 260
  17. 2015Explaining NonLinear Classification Decisions with Deep Taylor Decompositioncited by 248
  18. 2018Learning to Explain: An Information-Theoretic Perspective on Model Interpretationcited by 248
  19. 2016Understanding Neural Networks through Representation Erasurecited by 223
  20. 2020Neural Prototype Trees for Interpretable Fine-grained Image Recognitioncited by 213
  21. 2019Fixup Initialization: Residual Learning Without Normalizationcited by 194
  22. 2019Full-Gradient Representation for Neural Network Visualizationcited by 189
  23. 2022Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyondcited by 171
  24. 2017Visualizing Deep Neural Network Decisions: Prediction Difference Analysiscited by 166
  25. 2020Axiom-based Grad-CAM: Towards Accurate Visualization and Explanation of CNNscited by 151
  26. 2020Debugging Tests for Model Explanationscited by 146
  27. 2018Explaining Image Classifiers by Counterfactual Generationcited by 144
  28. 2019Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Groundedcited by 127
  29. 2019Sanity Checks for Saliency Metricscited by 119
  30. 2019Smooth Grad-CAM++: An Enhanced Inference Level Visualization Technique for Deep Convolutional Neural Network Modelscited by 118
  31. 2022A Consistent and Efficient Evaluation Strategy for Attribution Methodscited by 100
  32. 2018Interpreting CNNs via Decision Treescited by 89
  33. 2022B-cos Networks: Alignment is All We Need for Interpretabilitycited by 88
  34. 2019Robust and interpretable blind image denoising via bias-free convolutional neural networkscited by 87
  35. 2021The Out-of-Distribution Problem in Explainability and Search Methods for Feature Importance Explanationscited by 86
  36. 2021Convolutional Dynamic Alignment Networks for Interpretable Classificationscited by 54
  37. 2023FunnyBirds: A Synthetic Vision Dataset for a Part-Based Analysis of Explainable AI Methodscited by 49
  38. 2020IROF: a low resource evaluation metric for explanation methodscited by 45

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 4.

7 papers.

  1. 2025Birds look like cars: Adversarial analysis of intrinsically interpretable deep learningcited by 3
  2. 2025Beyond Accuracy: What Matters in Designing Well-Behaved Image Classification Models?cited by 1
  3. 2025Value bounds and Convergence Analysis for Averages of LRP attributionscited by 0
  4. 2026On the Fragility of Data Attribution When Learning Is Distributedcited by 0
  5. 2026Learning Quantifiable Visual Explanations Without Ground-Truthcited by 0
  6. 2026Improving Adversarial Robustness of Attribution via Implicit Regularizationcited by 0
  7. 2026Decomposition of Evidence, Contradiction, and Fragility in Perturbation Responsescited by 0