mlsanityData to 14 Oct 2026

Evaluating saliency methods on artificial data with different background types

arXiv 2112.04882 ↗First posted 9 Dec 2021eess.IV cs.CV cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Over the last years, many 'explainable artificial intelligence' (xAI) approaches have been developed, but these have not always been objectively evaluated. To evaluate the quality of heatmaps generated by various saliency methods, we developed a framework to generate artificial data with synthetic lesions and a known ground truth map. Using this framework, we evaluated two data sets with different backgrounds, Perlin noise and 2D brain MRI slices, and found that the heatmaps vary strongly between saliency methods and backgrounds. We strongly encourage further evaluation of saliency maps and xAI methods using this framework before applying these in clinical or other safety-critical settings.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 17 of its 31 references that are in the map. First posted on arXiv 9 Dec 2021.

Builds on

17 of 31 references.

References · 17

  1. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  2. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  3. 2013Visualizing and Understanding Convolutional Networkscited by 2,290
  4. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,552
  5. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
  6. 2014Striving for Simplicity: The All Convolutional Netcited by 990
  7. 2018A Benchmark for Interpretability Methods in Deep Neural Networkscited by 476
  8. 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 419
  9. 2019Unmasking Clever Hans Predictors and Assessing What Machines Really Learncited by 365
  10. 2019When Explanations Lie: Why Many Modified BP Attributions Failcited by 92
  11. 2019Benchmarking Attribution Methods with Relative Feature Importancecited by 89
  12. 2017Learning how to explain neural networks: PatternNet and PatternAttributioncited by 84
5 more
  1. 2018A Theoretical Explanation for Perplexing Behaviors of Backpropagation-based Visualizationscited by 60
  2. 2018iNNvestigate neural networks!cited by 58
  3. 2019Testing the robustness of attribution methods for convolutional neural networks in MRI-based Alzheimer's disease classificationcited by 25
  4. 2019Uncovering convolutional neural network decisions for diagnosing multiple sclerosis on conventional MRI using layer-wise relevance propagationcited by 21
  5. 2020Quantifying Explainability of Saliency Methods in Deep Neural Networks with a Synthetic Datasetcited by 14

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 0; 2024: 2.

3 papers.

  1. 2022Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyondcited by 171
  2. 2024Decoupling Pixel Flipping and Occlusion Strategy for Consistent XAI Benchmarkscited by 28
  3. 2024Explainable AI needs formalizationcited by 5