# NoiseCAM: Explainable AI for the Boundary Between Noise and Adversarial Attacks

arXiv: https://arxiv.org/abs/2303.06151 · HTML: https://mlsanity.com/paper/2303.06151/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 40 references that are in the map.
- First posted on arXiv 9 Mar 2023.

## Builds on (most cited first, 24 of 24)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks](https://mlsanity.com/paper/1710.11063/) (2017)
- [Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models](https://mlsanity.com/paper/1805.06605/) (2018)
- [DeepXplore: Automated Whitebox Testing of Deep Learning Systems](https://mlsanity.com/paper/1705.06640/) (2017)
- [Axiom-based Grad-CAM: Towards Accurate Visualization and Explanation of CNNs](https://mlsanity.com/paper/2008.02312/) (2020)
- [Towards Deep Neural Network Architectures Robust to Adversarial Examples](https://mlsanity.com/paper/1412.5068/) (2014)
- [Do Wider Neural Networks Really Help Adversarial Robustness?](https://mlsanity.com/paper/2010.01279/) (2020)
- [Medical image denoising using convolutional denoising autoencoders](https://mlsanity.com/paper/1608.04667/) (2016)
- [TensorFuzz: Debugging Neural Networks with Coverage-Guided Fuzzing](https://mlsanity.com/paper/1807.10875/) (2018)
- [Convergence of Adversarial Training in Overparametrized Neural Networks](https://mlsanity.com/paper/1906.07916/) (2019)
- [Interpolated Adversarial Training: Achieving Robust Neural Networks without Sacrificing Too Much Accuracy](https://mlsanity.com/paper/1906.06784/) (2019)
- [Towards Imperceptible and Robust Adversarial Example Attacks against Neural Networks](https://mlsanity.com/paper/1801.04693/) (2018)
- [PuVAE: A Variational Autoencoder to Purify Adversarial Examples](https://mlsanity.com/paper/1903.00585/) (2019)
- [Distant Domain Transfer Learning for Medical Imaging](https://mlsanity.com/paper/2012.06346/) (2020)
- [Revisiting Neuron Coverage Metrics and Quality of Deep Neural Networks](https://mlsanity.com/paper/2201.00191/) (2022)
- [Counter-Unmanned Aircraft System(s) (C-UAS): State of the Art, Challenges and Future Trends](https://mlsanity.com/paper/2008.12461/) (2020)
- [Machine Learning for the Detection and Identification of Internet of Things (IoT) Devices: A Survey](https://mlsanity.com/paper/2101.10181/) (2021)
- [Evaluating the performance of the LIME and Grad-CAM explanation methods on a LEGO multi-label image classification task](https://mlsanity.com/paper/2008.01584/) (2020)
- [Exploring Adversarial Attacks on Neural Networks: An Explainable Approach](https://mlsanity.com/paper/2303.06032/) (2023)
- [An integrated Auto Encoder-Block Switching defense approach to prevent adversarial attacks](https://mlsanity.com/paper/2203.10930/) (2022)

## Built on by (most cited first, 2 of 2)

- [A Survey on Verification and Validation, Testing and Evaluations of Neurosymbolic Artificial Intelligence](https://mlsanity.com/paper/2401.03188/) (2024)
- [Explainable AI for Comparative Analysis of Intrusion Detection Models](https://mlsanity.com/paper/2406.09684/) (2024)

## Abstract (arXiv)

Deep Learning (DL) and Deep Neural Networks (DNNs) are widely used in various domains. However, adversarial attacks can easily mislead a neural network and lead to wrong decisions. Defense mechanisms are highly preferred in safety-critical applications. In this paper, firstly, we use the gradient class activation map (GradCAM) to analyze the behavior deviation of the VGG-16 network when its inputs are mixed with adversarial perturbation or Gaussian noise. In particular, our method can locate vulnerable layers that are sensitive to adversarial perturbation and Gaussian noise. We also show that the behavior deviation of vulnerable layers can be used to detect adversarial examples. Secondly, we propose a novel NoiseCAM algorithm that integrates information from globally and pixel-level weighted class activation maps. Our algorithm is susceptible to adversarial perturbations and will not respond to Gaussian random noise mixed in the inputs. Third, we compare detecting adversarial examples using both behavior deviation and NoiseCAM, and we show that NoiseCAM outperforms behavior deviation modeling in its overall performance. Our work could provide a useful tool to defend against certain adversarial attacks on deep neural networks.