# Quantifying Explainability of Saliency Methods in Deep Neural Networks with a Synthetic Dataset

arXiv: https://arxiv.org/abs/2009.02899 · HTML: https://mlsanity.com/paper/2009.02899/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Sep 2020.

## Built on by (most cited first, 14 of 14)

- [Transparency of Deep Neural Networks for Medical Image Analysis: A Review of Interpretability Methods](https://mlsanity.com/paper/2111.02398/) (2021)
- [Benchmarking the Influence of Pre-training on Explanation Performance in MR Image Classification](https://mlsanity.com/paper/2306.12150/) (2023)
- [Structured Gradient-based Interpretations via Norm-Regularized Adversarial Training](https://mlsanity.com/paper/2404.04647/) (2024)
- [Toward Understanding the Disagreement Problem in Neural Network Feature Attribution](https://mlsanity.com/paper/2404.11330/) (2024)
- [Exploring the Trade-off Between Model Performance and Explanation Plausibility of Text Classifiers Using Human Rationales](https://mlsanity.com/paper/2404.03098/) (2024)
- [GECOBench: A Gender-Controlled Text Dataset and Benchmark for Quantifying Biases in Explanations](https://mlsanity.com/paper/2406.11547/) (2024)
- [Evaluating saliency methods on artificial data with different background types](https://mlsanity.com/paper/2112.04882/) (2021)
- [Forward Learning for Gradient-based Black-box Saliency Map Generation](https://mlsanity.com/paper/2403.15603/) (2024)
- [Exploring SAIG Methods for an Objective Evaluation of XAI](https://mlsanity.com/paper/2602.08715/) (2026)
- [Improving Deep Neural Network Classification Confidence using Heatmap-based eXplainable AI](https://mlsanity.com/paper/2201.00009/) (2021)
- [Validation of a Hospital Digital Twin with Machine Learning](https://mlsanity.com/paper/2303.04117/) (2023)
- [SHAMSUL: Systematic Holistic Analysis to investigate Medical Significance Utilizing Local interpretability methods in deep learning for chest radiography pathology prediction](https://mlsanity.com/paper/2307.08003/) (2023)
- [A Pseudo-Boolean Polynomials Approach for Image Edge Detection](https://mlsanity.com/paper/2308.15557/) (2023)
- [IMPACTX: improving model performance by appropriately constraining the training with teacher explanations](https://mlsanity.com/paper/2502.12222/) (2025)

## Abstract (arXiv)

Post-hoc analysis is a popular category in eXplainable artificial intelligence (XAI) study. In particular, methods that generate heatmaps have been used to explain the deep neural network (DNN), a black-box model. Heatmaps can be appealing due to the intuitive and visual ways to understand them but assessing their qualities might not be straightforward. Different ways to assess heatmaps' quality have their own merits and shortcomings. This paper introduces a synthetic dataset that can be generated adhoc along with the ground-truth heatmaps for more objective quantitative assessment. Each sample data is an image of a cell with easily recognized features that are distinguished from localization ground-truth mask, hence facilitating a more transparent assessment of different XAI methods. Comparison and recommendations are made, shortcomings are clarified along with suggestions for future research directions to handle the finer details of select post-hoc analysis methods. Furthermore, mabCAM is introduced as the heatmap generation method compatible with our ground-truth heatmaps. The framework is easily generalizable and uses only standard deep learning components.