# On the Connection Between Adversarial Robustness and Saliency Map Interpretability

arXiv: https://arxiv.org/abs/1905.04172 · HTML: https://mlsanity.com/paper/1905.04172/

## Key facts

- Cited by 80 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 May 2019.

## Built on by (most cited first, 30 of 80)

- [From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI](https://mlsanity.com/paper/2201.08164/) (2022)
- [Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks](https://mlsanity.com/paper/2207.13243/) (2022)
- [Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyond](https://mlsanity.com/paper/2103.10689/) (2021)
- [Robustness and Accuracy Could Be Reconcilable by (Proper) Definition](https://mlsanity.com/paper/2202.10103/) (2022)
- [Diffusion Visual Counterfactual Explanations](https://mlsanity.com/paper/2210.11841/) (2022)
- [Trustworthy AI: A Computational Perspective](https://mlsanity.com/paper/2107.06641/) (2021)
- [Robust Explainability: A Tutorial on Gradient-Based Attribution Methods for Deep Neural Networks](https://mlsanity.com/paper/2107.11400/) (2021)
- [Adversarial Counterfactual Visual Explanations](https://mlsanity.com/paper/2303.09962/) (2023)
- [Spurious Features Everywhere -- Large-Scale Detection of Harmful Spurious Features in ImageNet](https://mlsanity.com/paper/2212.04871/) (2022)
- [Fast Diffusion-Based Counterfactuals for Shortcut Removal and Generation](https://mlsanity.com/paper/2312.14223/) (2023)
- [Leave-one-out Unfairness](https://mlsanity.com/paper/2107.10171/) (2021)
- [A Comprehensive Survey on Trustworthy Recommender Systems](https://mlsanity.com/paper/2209.10117/) (2022)
- [Sparse Visual Counterfactual Explanations in Image Space](https://mlsanity.com/paper/2205.07972/) (2022)
- [Interpretability Benchmark for Evaluating Spatial Misalignment of Prototypical Parts Explanations](https://mlsanity.com/paper/2308.08162/) (2023)
- [What Makes a Good Explanation?: A Harmonized View of Properties of Explanations](https://mlsanity.com/paper/2211.05667/) (2022)
- [Robust Models Are More Interpretable Because Attributions Look Normal](https://mlsanity.com/paper/2103.11257/) (2021)
- [Attack to Fool and Explain Deep Networks](https://mlsanity.com/paper/2106.10606/) (2021)
- [Do Perceptually Aligned Gradients Imply Adversarial Robustness?](https://mlsanity.com/paper/2207.11378/) (2022)
- [Identifying Untrustworthy Predictions in Neural Networks by Geometric Gradient Analysis](https://mlsanity.com/paper/2102.12196/) (2021)
- [Adversarial Visual Robustness by Causal Intervention](https://mlsanity.com/paper/2106.09534/) (2021)
- [Fooling Explanations in Text Classifiers](https://mlsanity.com/paper/2206.03178/) (2022)
- [CLIPAG: Towards Generator-Free Text-to-Image Generation](https://mlsanity.com/paper/2306.16805/) (2023)
- [An Unsupervised Approach to Achieve Supervised-Level Explainability in Healthcare Records](https://mlsanity.com/paper/2406.08958/) (2024)
- [Unifying Model Explainability and Robustness for Joint Text Classification and Rationale Extraction](https://mlsanity.com/paper/2112.10424/) (2021)
- [How Does Frequency Bias Affect the Robustness of Neural Image Classifiers against Common Corruption and Adversarial Perturbations?](https://mlsanity.com/paper/2205.04533/) (2022)
- [When and How to Fool Explainable Models (and Humans) with Adversarial Examples](https://mlsanity.com/paper/2107.01943/) (2021)
- [Concept Alignment](https://mlsanity.com/paper/2401.08672/) (2024)
- [How explainable are adversarially-robust CNNs?](https://mlsanity.com/paper/2205.13042/) (2022)
- [Bayesian Neural Networks Avoid Encoding Complex and Perturbation-Sensitive Concepts](https://mlsanity.com/paper/2302.13095/) (2023)
- [ACAT: Adversarial Counterfactual Attention for Classification and Detection in Medical Imaging](https://mlsanity.com/paper/2303.15421/) (2023)

## Abstract (arXiv)

Recent studies on the adversarial vulnerability of neural networks have shown that models trained to be more robust to adversarial attacks exhibit more interpretable saliency maps than their non-robust counterparts. We aim to quantify this behavior by considering the alignment between input image and saliency map. We hypothesize that as the distance to the decision boundary grows,so does the alignment. This connection is strictly true in the case of linear models. We confirm these theoretical findings with experiments based on models trained with a local Lipschitz regularization and identify where the non-linear nature of neural networks weakens the relation.