mlsanityData to 7 Oct 2026

A Fresh Look at Sanity Checks for Saliency Maps

arXiv 2405.02383 ↗First posted 3 May 2024stat.ML cs.AI cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

The Model Parameter Randomisation Test (MPRT) is highly recognised in the eXplainable Artificial Intelligence (XAI) community due to its fundamental evaluative criterion: explanations should be sensitive to the parameters of the model they seek to explain. However, recent studies have raised several methodological concerns for the empirical interpretation of MPRT. In response, we propose two modifications to the original test: Smooth MPRT and Efficient MPRT. The former reduces the impact of noise on evaluation outcomes via sampling, while the latter avoids the need for biased similarity measurements by re-interpreting the test through the increase in explanation complexity after full model randomisation. Our experiments show that these modifications enhance the metric reliability, facilitating a more trustworthy deployment of explanation methods.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 39 of its 56 references that are in the map. First posted on arXiv 3 May 2024.

Builds on

39 of 56 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  6. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  7. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  8. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  9. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  10. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  11. 2018Sanity Checks for Saliency Mapscited by 1,149
  12. 2017SmoothGrad: removing noise by adding noisecited by 1,097
27 more
  1. 2014Striving for Simplicity: The All Convolutional Netcited by 983
  2. 2020Captum: A unified and generic model interpretability library for PyTorchcited by 542
  3. 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
  4. 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 416
  5. 2017Methods for Interpreting and Understanding Deep Neural Networkscited by 342
  6. 2019On the (In)fidelity and Sensitivity for Explanationscited by 309
  7. 2016Not Just a Black Box: Learning Important Features Through Propagating Activation Differencescited by 295
  8. 2015Explaining NonLinear Classification Decisions with Deep Taylor Decompositioncited by 248
  9. 2020Evaluating and Aggregating Feature-based Model Explanationscited by 185
  10. 2022Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyondcited by 171
  11. 2022A Consistent and Efficient Evaluation Strategy for Attribution Methodscited by 100
  12. 2019When Explanations Lie: Why Many Modified BP Attributions Failcited by 92
  13. 2022Which Explanation Should I Choose? A Function Approximation Perspective to Characterizing Post Hoc Explanationscited by 77
  14. 2020Understanding SSIMcited by 75
  15. 2020On quantitative aspects of model interpretabilitycited by 71
  16. 2022Framework for Evaluating Faithfulness of Local Explanationscited by 55
  17. 2021Software for Dataset-wide XAI: From Local Explanations to Global Insights with Zennit, CoRelAy, and ViRelAycited by 53
  18. 2022Rethinking Stability for Attribution-based Explanationscited by 52
  19. 2020IROF: a low resource evaluation metric for explanation methodscited by 45
  20. 2023The Meta-Evaluation Problem in Explainable AI: Identifying Reliable Estimators with MetaQuantuscited by 37
  21. 2021NoiseGrad: Enhancing Explanations by Introducing Stochasticity to Model Weightscited by 28
  22. 2023Finding the right XAI method -- A Guide for the Evaluation and Ranking of Explainable AI Methods in Climate Sciencecited by 25
  23. 2022Shortcomings of Top-Down Randomization-Based Sanity Checks for Evaluations of Deep Neural Network Explanationscited by 22
  24. 2021Revisiting Sanity Checks for Saliency Mapscited by 20
  25. 2021Investigating sanity checks for saliency maps with image and text classificationcited by 11
  26. 2018A Note about: Local Explanation Methods for Deep Neural Networks lack Sensitivity to Parameter Valuescited by 9
  27. 2024Rethinking Explainable Machine Learning as Applied Statisticscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 4; 2026: 1.

8 papers.

  1. 2025Bridging the Gap in XAI-Why Reliable Metrics Matter for Explainability and Compliancecited by 6
  2. 2025Formal Abductive Latent Explanations for Prototype-Based Networkscited by 5
  3. 2024From Flexibility to Manipulation: The Slippery Slope of XAI Evaluationcited by 4
  4. 2024OMENN: One Matrix to Explain Neural Networkscited by 2
  5. 2024REPEAT: Improving Uncertainty Estimation in Representation Learning Explainabilitycited by 1
  6. 2025P-TAME: Explain Any Image Classifier with Trained Perturbationscited by 0
  7. 2025Attention Trajectories as a Diagnostic Axis for Deep Reinforcement Learningcited by 0
  8. 2026Structural Compactness as a Complementary Criterion for Explanation Qualitycited by 0