# Fooling Network Interpretation in Image Classification

arXiv: https://arxiv.org/abs/1812.02843 · HTML: https://mlsanity.com/paper/1812.02843/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Dec 2018.

## Built on by (most cited first, 4 of 4)

- [AdvRain: Adversarial Raindrops to Attack Camera-based Smart Vision Systems](https://mlsanity.com/paper/2303.01338/) (2023)
- [ODDR: Outlier Detection & Dimension Reduction Based Defense Against Adversarial Patches](https://mlsanity.com/paper/2311.12084/) (2023)
- [ExAD: An Ensemble Approach for Explanation-based Adversarial Detection](https://mlsanity.com/paper/2103.11526/) (2021)
- [PASA: Attack Agnostic Unsupervised Adversarial Detection using Prediction & Attribution Sensitivity Analysis](https://mlsanity.com/paper/2404.10789/) (2024)

## Abstract (arXiv)

Deep neural networks have been shown to be fooled rather easily using adversarial attack algorithms. Practical methods such as adversarial patches have been shown to be extremely effective in causing misclassification. However, these patches are highlighted using standard network interpretation algorithms, thus revealing the identity of the adversary. We show that it is possible to create adversarial patches which not only fool the prediction, but also change what we interpret regarding the cause of the prediction. Moreover, we introduce our attack as a controlled setting to measure the accuracy of interpretation algorithms. We show this using extensive experiments for Grad-CAM interpretation that transfers to occluding patch interpretation as well. We believe our algorithms can facilitate developing more robust network interpretation tools that truly explain the network's underlying decision making process.