# Interpreting Adversarially Trained Convolutional Neural Networks

arXiv: https://arxiv.org/abs/1905.09797 · HTML: https://mlsanity.com/paper/1905.09797/

## Key facts

- Cited by 75 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 May 2019.

## Built on by (most cited first, 30 of 75)

- [Partial success in closing the gap between human and machine vision](https://mlsanity.com/paper/2106.07411/) (2021)
- [Advances in adversarial attacks and defenses in computer vision: A survey](https://mlsanity.com/paper/2108.00401/) (2021)
- [Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyond](https://mlsanity.com/paper/2103.10689/) (2021)
- [A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinking](https://mlsanity.com/paper/2302.14301/) (2023)
- [Revisiting Adversarial Robustness Distillation: Robust Soft Labels Make Student Better](https://mlsanity.com/paper/2108.07969/) (2021)
- [RobustART: Benchmarking Robustness on Architecture Design and Training Techniques](https://mlsanity.com/paper/2109.05211/) (2021)
- [Better Safe Than Sorry: Preventing Delusive Adversaries with Adversarial Training](https://mlsanity.com/paper/2102.04716/) (2021)
- [X-Adv: Physical Adversarial Object Attacks against X-ray Prohibited Item Detection](https://mlsanity.com/paper/2302.09491/) (2023)
- [Person Re-identification Method Based on Color Attack and Joint Defence](https://mlsanity.com/paper/2111.09571/) (2021)
- [A Survey of Robust Adversarial Training in Pattern Recognition: Fundamental, Theory, and Methodologies](https://mlsanity.com/paper/2203.14046/) (2022)
- [A Comprehensive Evaluation Framework for Deep Model Robustness](https://mlsanity.com/paper/2101.09617/) (2021)
- [Robust and Accurate Object Detection via Adversarial Learning](https://mlsanity.com/paper/2103.13886/) (2021)
- [Pre-trained Trojan Attacks for Visual Recognition](https://mlsanity.com/paper/2312.15172/) (2023)
- [Robust Unlearnable Examples: Protecting Data Against Adversarial Learning](https://mlsanity.com/paper/2203.14533/) (2022)
- [Provably Efficient Black-Box Action Poisoning Attacks Against Reinforcement Learning](https://mlsanity.com/paper/2110.04471/) (2021)
- [One-Pixel Shortcut: on the Learning Preference of Deep Neural Networks](https://mlsanity.com/paper/2205.12141/) (2022)
- [Interpretability Benchmark for Evaluating Spatial Misalignment of Prototypical Parts Explanations](https://mlsanity.com/paper/2308.08162/) (2023)
- [Can We Talk Models Into Seeing the World Differently?](https://mlsanity.com/paper/2403.09193/) (2024)
- [Does Robustness on ImageNet Transfer to Downstream Tasks?](https://mlsanity.com/paper/2204.03934/) (2022)
- [Tree Prompting: Efficient Task Adaptation without Fine-Tuning](https://mlsanity.com/paper/2310.14034/) (2023)
- [Attack to Fool and Explain Deep Networks](https://mlsanity.com/paper/2106.10606/) (2021)
- [A Unified Game-Theoretic Interpretation of Adversarial Robustness](https://mlsanity.com/paper/2111.03536/) (2021)
- [Exploring the Application of Large-scale Pre-trained Models on Adverse Weather Removal](https://mlsanity.com/paper/2306.09008/) (2023)
- [Towards Building More Robust Models with Frequency Bias](https://mlsanity.com/paper/2307.09763/) (2023)
- [DiG-IN: Diffusion Guidance for Investigating Networks -- Uncovering Classifier Differences Neuron Visualisations and Visual Counterfactual Explanations](https://mlsanity.com/paper/2311.17833/) (2023)
- [When and How to Fool Explainable Models (and Humans) with Adversarial Examples](https://mlsanity.com/paper/2107.01943/) (2021)
- [Adversarially Robust CLIP Models Can Induce Better (Robust) Perceptual Metrics](https://mlsanity.com/paper/2502.11725/) (2025)
- [MixACM: Mixup-Based Robustness Transfer via Distillation of Activated Channel Maps](https://mlsanity.com/paper/2111.05073/) (2021)
- [How explainable are adversarially-robust CNNs?](https://mlsanity.com/paper/2205.13042/) (2022)
- [Impact of Spatial Frequency Based Constraints on Adversarial Robustness](https://mlsanity.com/paper/2104.12679/) (2021)

## Abstract (arXiv)

We attempt to interpret how adversarially trained convolutional neural networks (AT-CNNs) recognize objects. We design systematic approaches to interpret AT-CNNs in both qualitative and quantitative ways and compare them with normally trained models. Surprisingly, we find that adversarial training alleviates the texture bias of standard CNNs when trained on object recognition tasks, and helps CNNs learn a more shape-biased representation. We validate our hypothesis from two aspects. First, we compare the salience maps of AT-CNNs and standard CNNs on clean images and images under different transformations. The comparison could visually show that the prediction of the two types of CNNs is sensitive to dramatically different types of features. Second, to achieve quantitative verification, we construct additional test datasets that destroy either textures or shapes, such as style-transferred version of clean data, saturated images and patch-shuffled ones, and then evaluate the classification accuracy of AT-CNNs and normal CNNs on these datasets. Our findings shed some light on why AT-CNNs are more robust than those normally trained ones and contribute to a better understanding of adversarial training over CNNs from an interpretation perspective.