# The shape and simplicity biases of adversarially robust ImageNet-trained CNNs

arXiv: https://arxiv.org/abs/2006.09373 · HTML: https://mlsanity.com/paper/2006.09373/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jun 2020.

## Built on by (most cited first, 14 of 14)

- [Partial success in closing the gap between human and machine vision](https://mlsanity.com/paper/2106.07411/) (2021)
- [Do Input Gradients Highlight Discriminative Features?](https://mlsanity.com/paper/2102.12781/) (2021)
- [ImageNet-Hard: The Hardest Images Remaining from a Study of the Power of Zoom and Spatial Biases in Image Classification](https://mlsanity.com/paper/2304.05538/) (2023)
- [DiG-IN: Diffusion Guidance for Investigating Networks -- Uncovering Classifier Differences Neuron Visualisations and Visual Counterfactual Explanations](https://mlsanity.com/paper/2311.17833/) (2023)
- [How explainable are adversarially-robust CNNs?](https://mlsanity.com/paper/2205.13042/) (2022)
- [Inverting Adversarially Robust Networks for Image Synthesis](https://mlsanity.com/paper/2106.06927/) (2021)
- [One Wave To Explain Them All: A Unifying Perspective On Feature Attribution](https://mlsanity.com/paper/2410.01482/) (2024)
- [Shape Bias and Robustness Evaluation via Cue Decomposition for Image Classification and Segmentation](https://mlsanity.com/paper/2503.12453/) (2025)
- [Connecting metrics for shape-texture knowledge in computer vision](https://mlsanity.com/paper/2301.10608/) (2023)
- [Layerwise complexity-matched learning yields an improved model of cortical area V2](https://mlsanity.com/paper/2312.11436/) (2023)
- [Err on the Side of Texture: Texture Bias on Real Data](https://mlsanity.com/paper/2412.10597/) (2024)
- [On the Properties of Adversarially-Trained CNNs](https://mlsanity.com/paper/2203.09243/) (2022)
- [CPST: Comprehension-Preserving Style Transfer for Multi-Modal Narratives](https://mlsanity.com/paper/2312.08695/) (2023)
- [Noise Stability of Transformer Models](https://mlsanity.com/paper/2602.08287/) (2026)

## Abstract (arXiv)

Increasingly more similarities between human vision and convolutional neural networks (CNNs) have been revealed in the past few years. Yet, vanilla CNNs often fall short in generalizing to adversarial or out-of-distribution (OOD) examples which humans demonstrate superior performance. Adversarial training is a leading learning algorithm for improving the robustness of CNNs on adversarial and OOD data; however, little is known about the properties, specifically the shape bias and internal features learned inside adversarially-robust CNNs. In this paper, we perform a thorough, systematic study to understand the shape bias and some internal mechanisms that enable the generalizability of AlexNet, GoogLeNet, and ResNet-50 models trained via adversarial training. We find that while standard ImageNet classifiers have a strong texture bias, their R counterparts rely heavily on shapes. Remarkably, adversarial training induces three simplicity biases into hidden neurons in the process of "robustifying" CNNs. That is, each convolutional neuron in R networks often changes to detecting (1) pixel-wise smoother patterns, i.e., a mechanism that blocks high-frequency noise from passing through the network; (2) more lower-level features i.e. textures and colors (instead of objects);and (3) fewer types of inputs. Our findings reveal the interesting mechanisms that made networks more adversarially robust and also explain some recent findings e.g., why R networks benefit from a much larger capacity (Xie et al. 2020) and can act as a strong image prior in image synthesis (Santurkar et al. 2019).