# How to Manipulate CNNs to Make Them Lie: the GradCAM Case

arXiv: https://arxiv.org/abs/1907.10901 · HTML: https://mlsanity.com/paper/1907.10901/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jul 2019.

## Built on by (most cited first, 12 of 12)

- [Logic-Based Explainability in Machine Learning](https://mlsanity.com/paper/2211.00541/) (2022)
- [Adversarial attacks and defenses in explainable artificial intelligence: A survey](https://mlsanity.com/paper/2306.06123/) (2023)
- [Don't trust your eyes: on the (un)reliability of feature visualizations](https://mlsanity.com/paper/2306.04719/) (2023)
- [On Computing Probabilistic Abductive Explanations](https://mlsanity.com/paper/2212.05990/) (2022)
- [Making Corgis Important for Honeycomb Classification: Adversarial Attacks on Concept-based Explainability Tools](https://mlsanity.com/paper/2110.07120/) (2021)
- [A Rigorous Study Of The Deep Taylor Decomposition](https://mlsanity.com/paper/2211.08425/) (2022)
- [Birds look like cars: Adversarial analysis of intrinsically interpretable deep learning](https://mlsanity.com/paper/2503.08636/) (2025)
- [Attribution Explanations for Deep Neural Networks: A Theoretical Perspective](https://mlsanity.com/paper/2508.07636/) (2025)
- [Explainability of Deep Learning models for Urban Space perception](https://mlsanity.com/paper/2208.13555/) (2022)
- [BI-LAVA: Biocuration with Hierarchical Image Labeling through Active Learning and Visual Analysis](https://mlsanity.com/paper/2308.08003/) (2023)
- [Impact of Adversarial Attacks on Deep Learning Model Explainability](https://mlsanity.com/paper/2412.11119/) (2024)
- [Accuracy is Not Enough: Poisoning Interpretability in Federated Learning via Color Skew](https://mlsanity.com/paper/2511.13535/) (2025)

## Abstract (arXiv)

Recently many methods have been introduced to explain CNN decisions. However, it has been shown that some methods can be sensitive to manipulation of the input. We continue this line of work and investigate the explanation method GradCAM. Instead of manipulating the input, we consider an adversary that manipulates the model itself to attack the explanation. By changing weights and architecture, we demonstrate that it is possible to generate any desired explanation, while leaving the model's accuracy essentially unchanged. This illustrates that GradCAM cannot explain the decision of every CNN and provides a proof of concept showing that it is possible to obfuscate the inner workings of a CNN. Finally, we combine input and model manipulation. To this end we put a backdoor in the network: the explanation is correct unless there is a specific pattern present in the input, which triggers a malicious explanation. Our work raises new security concerns, especially in settings where explanations of models may be used to make decisions, such as in the medical domain.