mlsanityData to 14 Oct 2026

Decoupling Pixel Flipping and Occlusion Strategy for Consistent XAI Benchmarks

arXiv 2401.06654 ↗First posted 12 Jan 2024cs.CV cs.AI cs.LGcited by 28

Not on the map. Why?

Abstract (arXiv)

Feature removal is a central building block for eXplainable AI (XAI), both for occlusion-based explanations (Shapley values) as well as their evaluation (pixel flipping, PF). However, occlusion strategies can vary significantly from simple mean replacement up to inpainting with state-of-the-art diffusion models. This ambiguity limits the usefulness of occlusion-based approaches. For example, PF benchmarks lead to contradicting rankings. This is amplified by competing PF measures: Features are either removed starting with most influential first (MIF) or least influential first (LIF). This study proposes two complementary perspectives to resolve this disagreement problem. Firstly, we address the common criticism of occlusion-based XAI, that artificial samples lead to unreliable model evaluations. We propose to measure the reliability by the R(eference)-Out-of-Model-Scope (OMS) score. The R-OMS score enables a systematic comparison of occlusion strategies and resolves the disagreement problem by grouping consistent PF rankings. Secondly, we show that the insightfulness of MIF and LIF is conversely dependent on the R-OMS score. To leverage this, we combine the MIF and LIF measures into the symmetric relevance gain (SRG) measure. This breaks the inherent connection to the underlying occlusion strategy and leads to consistent rankings. This resolves the disagreement problem, which we verify for a set of 40 different occlusion strategies.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Builds on 73 of its 106 references that are in the map. First posted on arXiv 12 Jan 2024.

Builds on

73 of 106 references.

References · 73

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2023Segment Anythingcited by 8,234
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  5. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  6. 2013Visualizing and Understanding Convolutional Networkscited by 2,290
  7. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,430
  8. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
  9. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  10. 2018Sanity Checks for Saliency Mapscited by 1,156
  11. 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 838
  12. 2020Captum: A unified and generic model interpretability library for PyTorchcited by 547
38 more
  1. 2018A Benchmark for Interpretability Methods in Deep Neural Networkscited by 476
  2. 2019Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methodscited by 443
  3. 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 419
  4. 2021ResNet strikes back: An improved training procedure in timmcited by 368
  5. 2019Unmasking Clever Hans Predictors and Assessing What Machines Really Learncited by 365
  6. 2017Methods for Interpreting and Understanding Deep Neural Networkscited by 345
  7. 2019The many Shapley values for model explanationcited by 344
  8. 2019On the (In)fidelity and Sensitivity for Explanationscited by 310
  9. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 302
  10. 2023Inpaint Anything: Segment Anything Meets Image Inpaintingcited by 233
  11. 2020Explaining by Removing: A Unified Framework for Model Explanationcited by 218
  12. 2019Explanations can be manipulated and geometry is to blamecited by 216
  13. 2016Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Modelscited by 213
  14. 2020Problems with Shapley-value-based explanations as feature importance measurescited by 200
  15. 2020Evaluating and Aggregating Feature-based Model Explanationscited by 188
  16. 2022Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyondcited by 171
  17. 2017Visualizing Deep Neural Network Decisions: Prediction Difference Analysiscited by 166
  18. 2022The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspectivecited by 159
  19. 2021A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challengescited by 154
  20. 2018Explaining Image Classifiers by Counterfactual Generationcited by 144
  21. 2022From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagationcited by 141
  22. 2020Ground Truth Evaluation of Neural Network Explanations with CLEVR-XAIcited by 126
  23. 2019Sanity Checks for Saliency Metricscited by 119
  24. 2019XRAI: Better Attributions Through Regionscited by 116
  25. 2020Explaining Explanations: Axiomatic Feature Interactions for Deep Networkscited by 103
  26. 2022A Consistent and Efficient Evaluation Strategy for Attribution Methodscited by 102
  27. 2019Benchmarking Attribution Methods with Relative Feature Importancecited by 89
  28. 2019Unrestricted Permutation forces Extrapolation: Variable Importance Requires at least One More Model, or There Is No Free Variable Importancecited by 87
  29. 2021The Out-of-Distribution Problem in Explainability and Search Methods for Feature Importance Explanationscited by 87
  30. 2021e-ViL: A Dataset and Benchmark for Natural Language Explanations in Vision-Language Taskscited by 84
  31. 2020How does this interaction affect me? Interpretable attribution for feature interactionscited by 73
  32. 2020Fairwashing Explanations with Off-Manifold Detergentcited by 72
  33. 2022From Shapley Values to Generalized Additive Models and backcited by 60
  34. 2021Software for Dataset-wide XAI: From Local Explanations to Global Insights with Zennit, CoRelAy, and ViRelAycited by 55
  35. 2023Multi-dimensional concept discovery (MCD): A unifying framework with completeness guaranteescited by 50
  36. 2020IROF: a low resource evaluation metric for explanation methodscited by 46
  37. 2020Towards Robust Explanations for Deep Neural Networkscited by 43
  38. 2022Metrics for saliency map evaluation of deep learning explanation methodscited by 40

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 12; 2025: 8; 2026: 8.

28 papers.

Sort:

By area:

  1. 2024AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformerscited by 120
  2. 2024MambaLRP: Explaining Selective State Space Sequence Modelscited by 32
  3. 2024xMIL: Insightful Explanations for Multiple Instance Learning in Histopathologycited by 16
  4. 2024Pruning By Explaining Revisited: Optimizing Attribution Methods to Prune CNNs and Transformerscited by 11
  5. 2025Attention Consistency for LLMs Explanationcited by 11
  6. 2024Beyond Pixels: Enhancing LIME with Hierarchical Features and Segmentation Foundation Modelscited by 8
  7. 2024Towards Symbolic XAI -- Explanation Through Human Understandable Logical Relationships Between Featurescited by 7
  8. 2025Towards Desiderata-Driven Design of Visual Counterfactual Explainerscited by 7
  9. 2025Explaining Similarity in Vision-Language Encoders with Weighted Banzhaf Interactionscited by 6
  10. 2024Explainable AI needs formalizationcited by 5
  11. 2024LibraGrad: Balancing Gradient Flow for Universally Better Vision Transformer Attributionscited by 5
  12. 2024Explaining Time Series Classification Predictions via Causal Attributionscited by 4
16 more
  1. 2024From Flexibility to Manipulation: The Slippery Slope of XAI Evaluationcited by 4
  2. 2024Segmentation and Smoothing Affect Explanation Quality More Than the Choice of Perturbation-based XAI Method for Image Explanationscited by 2
  3. 2025Wasserstein Distances Made Explainable: Insights Into Dataset Shifts and Transport Phenomenacited by 2
  4. 2026Feature salience - not task-informativeness - drives machine learning model explanationscited by 2
  5. 2026Beyond Attention Heatmaps: How to Get Better Explanations for Multiple Instance Learning Models in Histopathologycited by 2
  6. 2025There is More to Attention: Statistical Filtering Enhances Explanations in Vision Transformerscited by 1
  7. 2025Back to the Baseline: Examining Baseline Effects on Explainability Metricscited by 1
  8. 2026xaitimesynth: A Python Package for Evaluating Attribution Methods for Time Series with Synthetic Ground Truthcited by 1
  9. 2026Measuring Cross-Modal Synergy: A Benchmark for VLM Explainabilitycited by 1
  10. 2024XpertAI: uncovering regression model strategies for sub-manifoldscited by 0
  11. 2025Uncovering the Structure of Explanation Quality with Spectral Analysiscited by 0
  12. 2025Always Keep Your Promises: A Model-Agnostic Attribution Algorithm for Neural Networkscited by 0
  13. 2026Does Explanation Correctness Matter? Linking Computational XAI Evaluation to Human Understandingcited by 0
  14. 2026XtrAIn: Training-Guided Occlusion for Feature Attributioncited by 0
  15. 2026ResLRP: The Role of Residual Cancellation in Attribution Instability in Vision Transformerscited by 0
  16. 2026Mask-Induced Displacement in Audio XAI via Logit Trajectory Decompositioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024IBO: Inpainting-Based Occlusion to Enhance Explainable Artificial Intelligence Evaluation in Histopathology
  2. 2023Explainable GeoAI: Can saliency maps help interpret artificial intelligence's learning process? An empirical study on natural feature detection
  3. 2025A Novel Framework for Automated Explain Vision Model Using Vision-Language Models
  4. 2026FlowCF: Sparse Counterfactual Explanations for Mixed-Type Tabular Data using Flow Matching
  5. 2024Benchmarking XAI Explanations with Human-Aligned Evaluations
  6. 2025Beyond the Visible: Benchmarking Occlusion Perception in Multimodal Large Language Models
  7. 2023Diffusion-based Visual Counterfactual Explanations -- Towards Systematic Quantitative Evaluation
  8. 2024Decompose the model: Mechanistic interpretability in image models with Generalized Integrated Gradients (GIG)