mlsanityData to 7 Oct 2026

A Consistent and Efficient Evaluation Strategy for Attribution Methods

arXiv 2202.00449 ↗First posted 1 Feb 2022cs.CV cs.LGcited by 100

Not on the map. Why?

Abstract (arXiv)

With a variety of local feature attribution methods being proposed in recent years, follow-up work suggested several evaluation strategies. To assess the attribution quality across different attribution techniques, the most popular among these evaluation strategies in the image domain use pixel perturbations. However, recent advances discovered that different evaluation strategies produce conflicting rankings of attribution methods and can be prohibitively expensive to compute. In this work, we present an information-theoretic analysis of evaluation strategies based on pixel perturbations. Our findings reveal that the results are strongly affected by information leakage through the shape of the removed pixels as opposed to their actual values. Using our theoretical insights, we propose a novel evaluation framework termed Remove and Debias (ROAD) which offers two contributions: First, it mitigates the impact of the confounders, which entails higher consistency among evaluation strategies. Second, ROAD does not require the computationally expensive retraining step and saves up to 99% in computational costs compared to the state-of-the-art. We release our source code at https://github.com/tleemann/road_evaluation.

Key facts. Cited by 100 papers in the map (data through 7 Oct 2026). Builds on 37 of its 53 references that are in the map. First posted on arXiv 1 Feb 2022.

Builds on

37 of 53 references.

References · 37

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  5. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  6. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  7. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  8. 2018Sanity Checks for Saliency Mapscited by 1,149
  9. 2017SmoothGrad: removing noise by adding noisecited by 1,097
  10. 2014Striving for Simplicity: The All Convolutional Netcited by 983
  11. 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 825
  12. 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 611
25 more
  1. 2018A Benchmark for Interpretability Methods in Deep Neural Networkscited by 467
  2. 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 416
  3. 2019A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAIcited by 403
  4. 2018GAIN: Missing Data Imputation using Generative Adversarial Netscited by 349
  5. 2019On the (In)fidelity and Sensitivity for Explanationscited by 309
  6. 2020Explaining by Removing: A Unified Framework for Model Explanationcited by 218
  7. 2019Explainable Machine Learning in Deploymentcited by 210
  8. 2019Full-Gradient Representation for Neural Network Visualizationcited by 189
  9. 2020Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior?cited by 189
  10. 2020Evaluating and Aggregating Feature-based Model Explanationscited by 185
  11. 2020Debugging Tests for Model Explanationscited by 146
  12. 2020Making deep neural networks right for the right scientific reasons by interacting with their explanationscited by 141
  13. 2019Sanity Checks for Saliency Metricscited by 119
  14. 2018L-Shapley and C-Shapley: Efficient Model Interpretation for Structured Datacited by 100
  15. 2020On quantitative aspects of model interpretabilitycited by 71
  16. 2015A Review of Feature Selection Methods Based on Mutual Informationcited by 67
  17. 2020Attribution in Scale and Spacecited by 61
  18. 2021Have We Learned to Explain?: How Interpretability Methods Can Learn to Encode Predictions in their Interpretationscited by 59
  19. 2021Do Input Gradients Highlight Discriminative Features?cited by 52
  20. 2021On Baselines for Local Feature Attributionscited by 29
  21. 2020How Good is your Explanation? Algorithmic Stability Measures to Assess the Quality of Explanations for Deep Neural Networkscited by 27
  22. 2021MIMIC-IF: Interpretability and Fairness Evaluation of Deep Learning Models on MIMIC-IV Datasetcited by 15
  23. 2020A Baseline for Shapley Values in MLPs: from Missingness to Neutralitycited by 11
  24. 2021Towards Rigorous Interpretations: a Formalisation of Feature Attributioncited by 11
  25. 2020Making Neural Networks Interpretable with Attribution: Application to Implicit Signals Predictioncited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 21; 2024: 31; 2025: 20; 2026: 22.

100 papers.

Sort:

By area:

  1. 2022Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanationscited by 84
  2. 2023The Meta-Evaluation Problem in Explainable AI: Identifying Reliable Estimators with MetaQuantuscited by 37
  3. 2023Visual Explanations of Image-Text Representations via Multi-Modal Information Bottleneck Attributioncited by 34
  4. 2024Decoupling Pixel Flipping and Occlusion Strategy for Consistent XAI Benchmarkscited by 28
  5. 2023Finding the right XAI method -- A Guide for the Evaluation and Ranking of Explainable AI Methods in Climate Sciencecited by 25
  6. 2022On the Trade-Off between Actionable Explanations and the Right to be Forgottencited by 20
  7. 2023CoRTX: Contrastive Framework for Real-time Explanationcited by 20
  8. 2024On the Faithfulness of Vision Transformer Explanationscited by 19
  9. 2024F-Fidelity: A Robust Framework for Faithfulness Evaluation of Explainable AIcited by 17
  10. 2023The Co-12 Recipe for Evaluating Interpretable Part-Prototype Image Classifierscited by 15
  11. 2023Assessing Fidelity in XAI post-hoc techniques: A Comparative Study with Ground Truth Explanations Datasetscited by 15
  12. 2024Navigating the Maze of Explainable AI: A Systematic Approach to Evaluating Methods and Metricscited by 15
38 more
  1. 2023Stability Guarantees for Feature Attributions with Multiplicative Smoothingcited by 14
  2. 2024Interpreting and Analysing CLIP's Zero-Shot Image Classification via Mutual Knowledgecited by 12
  3. 2023Don't be fooled: label leakage in explanation methods and the importance of their quantitative evaluationcited by 11
  4. 2023Feature Perturbation Augmentation for Reliable Evaluation of Importance Estimators in Neural Networkscited by 9
  5. 2024Sanity Checks Revisited: An Exploration to Repair the Model Parameter Randomisation Testcited by 9
  6. 2025Probabilistic Stability Guarantees for Feature Attributionscited by 9
  7. 2024A Fresh Look at Sanity Checks for Saliency Mapscited by 8
  8. 2024Attention Mechanisms Don't Learn Additive Models: Rethinking Feature Importance for Transformerscited by 8
  9. 2024Benchmarking XAI Explanations with Human-Aligned Evaluationscited by 8
  10. 2024Benchmarking the Attribution Quality of Vision Modelscited by 7
  11. 2023Efficient GNN Explanation via Learning Removal-based Attributioncited by 6
  12. 2024KPCA-CAM: Visual Explainability of Deep Computer Vision Models using Kernel PCAcited by 6
  13. 2025Bridging the Gap in XAI-Why Reliable Metrics Matter for Explainability and Compliancecited by 6
  14. 2025Generalizable and Explainable Deep Learning for Medical Image Computing: An Overviewcited by 6
  15. 2025Comprehensive Information Bottleneck for Unveiling Universal Attribution to Interpret Vision Transformerscited by 6
  16. 2025GrAInS: Gradient-based Attribution for Inference-Time Steering of LLMs and VLMscited by 6
  17. 2024GECOBench: A Gender-Controlled Text Dataset and Benchmark for Quantifying Biases in Explanationscited by 5
  18. 2024On the Evaluation Consistency of Attribution-based Explanationscited by 5
  19. 2022Sensing accident-prone features in urban scenes for proactive driving and accident preventioncited by 4
  20. 2023A Survey of Explainable AI in Deep Visual Modeling: Methods and Metricscited by 4
  21. 2024T-TAME: Trainable Attention Mechanism for Explaining Convolutional Networks and Vision Transformerscited by 4
  22. 2024Expected Grad-CAM: Towards gradient faithfulnesscited by 4
  23. 2024Explainable AI needs formalizationcited by 4
  24. 2024From Flexibility to Manipulation: The Slippery Slope of XAI Evaluationcited by 4
  25. 2022E Pluribus Unum Interpretable Convolutional Neural Networkscited by 3
  26. 2023AttributionLab: Faithfulness of Feature Attribution Under Controllable Environmentscited by 3
  27. 2023A Comprehensive and Reliable Feature Attribution Method: Double-sided Remove and Reconstruct (DoRaR)cited by 3
  28. 2024Geometric Remove-and-Retrain (GOAR): Coordinate-Invariant eXplainable AI Assessmentcited by 3
  29. 2024Counterfactuals As a Means for Evaluating Faithfulness of Attribution Methods in Autoregressive Language Modelscited by 3
  30. 2024IBO: Inpainting-Based Occlusion to Enhance Explainable Artificial Intelligence Evaluation in Histopathologycited by 3
  31. 2023Analyzing Effects of Mixed Sample Data Augmentation on Model Interpretabilitycited by 2
  32. 2023On Pitfalls of textitRemOve-And-Retrain: Data Processing Inequality Perspectivecited by 2
  33. 2023Looking deeper into interpretable deep learning in neuroimaging: a comprehensive surveycited by 2
  34. 2023Fast Shapley Value Estimation: A Unified Approachcited by 2
  35. 2024CA-Stream: Attention-based pooling for interpretable image recognitioncited by 2
  36. 2024A Backdoor-based Explainable AI Benchmark for High Fidelity Evaluation of Attributionscited by 2
  37. 2024Axiomatic Explainer Globalness via Optimal Transportcited by 2
  38. 2024Derivative-Free Diffusion Manifold-Constrained Gradient for Unified XAIcited by 2

50 most cited of 100 All 100 (JSON)