# Shap-CAM: Visual Explanations for Convolutional Neural Networks based on Shapley Value

arXiv: https://arxiv.org/abs/2208.03608 · HTML: https://mlsanity.com/paper/2208.03608/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 36 references that are in the map.
- First posted on arXiv 7 Aug 2022.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks](https://mlsanity.com/paper/1710.11063/) (2017)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [Interpretable Explanations of Black Boxes by Meaningful Perturbation](https://mlsanity.com/paper/1704.03296/) (2017)
- [Network In Network](https://mlsanity.com/paper/1312.4400/) (2013)
- [Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks](https://mlsanity.com/paper/1910.01279/) (2019)
- [Real Time Image Saliency for Black Box Classifiers](https://mlsanity.com/paper/1705.07857/) (2017)
- [Explaining Image Classifiers by Counterfactual Generation](https://mlsanity.com/paper/1807.08024/) (2018)
- [Smooth Grad-CAM++: An Enhanced Inference Level Visualization Technique for Deep Convolutional Neural Network Models](https://mlsanity.com/paper/1908.01224/) (2019)
- [Fooling Neural Network Interpretations via Adversarial Model Manipulation](https://mlsanity.com/paper/1902.02041/) (2019)
- [Local Explanation Methods for Deep Neural Networks Lack Sensitivity to Parameter Values](https://mlsanity.com/paper/1810.03307/) (2018)
- [Contextual Explanation Networks](https://mlsanity.com/paper/1705.10301/) (2017)

## Built on by (most cited first, 23 of 23)

- [Adversarial attacks and defenses in explainable artificial intelligence: A survey](https://mlsanity.com/paper/2306.06123/) (2023)
- [Adversarial Counterfactual Visual Explanations](https://mlsanity.com/paper/2303.09962/) (2023)
- [On the Faithfulness of Vision Transformer Explanations](https://mlsanity.com/paper/2404.01415/) (2024)
- [Keep the Faith: Faithful Explanations in Convolutional Neural Networks for Case-Based Reasoning](https://mlsanity.com/paper/2312.09783/) (2023)
- [Feature Map Convergence Evaluation for Functional Module](https://mlsanity.com/paper/2405.04041/) (2024)
- [Explaining Object Detectors via Collective Contribution of Pixels](https://mlsanity.com/paper/2412.00666/) (2024)
- [BSED: Baseline Shapley-Based Explainable Detector](https://mlsanity.com/paper/2308.07490/) (2023)
- [ShapeX: Shapelet-Driven Post Hoc Explanations for Time Series Classification Models](https://mlsanity.com/paper/2510.20084/) (2025)
- [Explainable AI is Responsible AI: How Explainability Creates Trustworthy and Socially Responsible Artificial Intelligence](https://mlsanity.com/paper/2312.01555/) (2023)
- [A Comprehensive Study of Shapley Value in Data Analytics](https://mlsanity.com/paper/2412.01460/) (2024)
- [OMENN: One Matrix to Explain Neural Networks](https://mlsanity.com/paper/2412.02399/) (2024)
- [Towards Fine-Grained Interpretability: Counterfactual Explanations for Misclassification with Saliency Partition](https://mlsanity.com/paper/2511.07974/) (2025)
- [MaskDiME: Adaptive Masked Diffusion for Precise and Efficient Visual Counterfactual Explanations](https://mlsanity.com/paper/2602.18792/) (2026)
- [Retinal Fundus Multi-Disease Image Classification using Hybrid CNN-Transformer-Ensemble Architectures](https://mlsanity.com/paper/2503.21465/) (2025)
- [ProtoSolo: Interpretable Image Classification via Single-Prototype Activation](https://mlsanity.com/paper/2506.19808/) (2025)
- [CE-FAM: Concept-Based Explanation via Fusion of Activation Maps](https://mlsanity.com/paper/2509.23849/) (2025)
- [PCEvE: Part Contribution Evaluation Based Model Explanation for Human Figure Drawing Assessment and Beyond](https://mlsanity.com/paper/2409.18260/) (2024)
- [Hierarchical Feature-level Reverse Propagation for Post-Training Neural Networks](https://mlsanity.com/paper/2506.07188/) (2025)
- [Grad-CAM for Visualizing Attention Regions of PCA and SVM Layers in Convolutional Neural Networks](https://mlsanity.com/paper/2508.11880/) (2025)
- [How can we trust opaque systems? Criteria for robust explanations in XAI](https://mlsanity.com/paper/2508.12623/) (2025)
- [Desensitizing for Improving Corruption Robustness in Point Cloud Classification through Adversarial Training](https://mlsanity.com/paper/2509.23010/) (2025)
- [Local Additive Feature Attribution: A Mathematical Taxonomy and Reporting Checklist](https://mlsanity.com/paper/2607.14271/) (2026)
- [Do CNNs Internally Represent Real and Fake Images Differently? A Hidden-Layer Analysis](https://mlsanity.com/paper/2608.14729/) (2026)

## Abstract (arXiv)

Explaining deep convolutional neural networks has been recently drawing increasing attention since it helps to understand the networks' internal operations and why they make certain decisions. Saliency maps, which emphasize salient regions largely connected to the network's decision-making, are one of the most common ways for visualizing and analyzing deep networks in the computer vision community. However, saliency maps generated by existing methods cannot represent authentic information in images due to the unproven proposals about the weights of activation maps which lack solid theoretical foundation and fail to consider the relations between each pixel. In this paper, we develop a novel post-hoc visual explanation method called Shap-CAM based on class activation mapping. Unlike previous gradient-based approaches, Shap-CAM gets rid of the dependence on gradients by obtaining the importance of each pixel through Shapley value. We demonstrate that Shap-CAM achieves better visual performance and fairness for interpreting the decision making process. Our approach outperforms previous methods on both recognition and localization tasks.