# Interpretation of Neural Networks is Susceptible to Universal Adversarial Perturbations

arXiv: https://arxiv.org/abs/2212.03095 · HTML: https://mlsanity.com/paper/2212.03095/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 28 references that are in the map.
- First posted on arXiv 30 Nov 2022.

## Builds on (most cited first, 14 of 14)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Universal adversarial perturbations](https://mlsanity.com/paper/1610.08401/) (2016)
- [Interpretation of Neural Networks is Fragile](https://mlsanity.com/paper/1710.10547/) (2017)
- [Generative Adversarial Perturbations](https://mlsanity.com/paper/1712.02328/) (2017)
- [Fooling Neural Network Interpretations via Adversarial Model Manipulation](https://mlsanity.com/paper/1902.02041/) (2019)
- [Universal Adversarial Training](https://mlsanity.com/paper/1811.11304/) (2018)
- [Art of singular vectors and universal adversarial perturbations](https://mlsanity.com/paper/1709.03582/) (2017)
- [Real-time, Universal, and Robust Adversarial Attacks Against Speaker Recognition Systems](https://mlsanity.com/paper/2003.02301/) (2020)
- [Attack on practical speaker verification system using universal adversarial perturbations](https://mlsanity.com/paper/2105.09022/) (2021)
- [Efficient universal shuffle attack for visual object tracking](https://mlsanity.com/paper/2203.06898/) (2022)

## Built on by (most cited first, 4 of 4)

- [Robust Decentralized Learning with Local Updates and Gradient Tracking](https://mlsanity.com/paper/2405.00965/) (2024)
- [Exploring Cross-model Neuronal Correlations in the Context of Predicting Model Performance and Generalizability](https://mlsanity.com/paper/2408.08448/) (2024)
- [Attack on Scene Flow using Point Clouds](https://mlsanity.com/paper/2404.13621/) (2024)
- [On the Fragility of AI-Based Channel Decoders under Small Channel Perturbations](https://mlsanity.com/paper/2602.01582/) (2026)

## Abstract (arXiv)

Interpreting neural network classifiers using gradient-based saliency maps has been extensively studied in the deep learning literature. While the existing algorithms manage to achieve satisfactory performance in application to standard image recognition datasets, recent works demonstrate the vulnerability of widely-used gradient-based interpretation schemes to norm-bounded perturbations adversarially designed for every individual input sample. However, such adversarial perturbations are commonly designed using the knowledge of an input sample, and hence perform sub-optimally in application to an unknown or constantly changing data point. In this paper, we show the existence of a Universal Perturbation for Interpretation (UPI) for standard image datasets, which can alter a gradient-based feature map of neural networks over a significant fraction of test samples. To design such a UPI, we propose a gradient-based optimization method as well as a principal component analysis (PCA)-based approach to compute a UPI which can effectively alter a neural network's gradient-based interpretation on different samples. We support the proposed UPI approaches by presenting several numerical results of their successful applications to standard image datasets.