# Shape or Texture: Understanding Discriminative Features in CNNs

arXiv: https://arxiv.org/abs/2101.11604 · HTML: https://mlsanity.com/paper/2101.11604/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 33 references that are in the map.
- First posted on arXiv 27 Jan 2021.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Network Dissection: Quantifying Interpretability of Deep Visual Representations](https://mlsanity.com/paper/1704.05796/) (2017)
- [On Variational Bounds of Mutual Information](https://mlsanity.com/paper/1905.06922/) (2019)
- [On Mutual Information Maximization for Representation Learning](https://mlsanity.com/paper/1907.13625/) (2019)
- [Exploring Self-attention for Image Recognition](https://mlsanity.com/paper/2004.13621/) (2020)
- [Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet](https://mlsanity.com/paper/1904.00760/) (2019)
- [Formal Limitations on the Measurement of Mutual Information](https://mlsanity.com/paper/1811.04251/) (2018)
- [Net2Vec: Quantifying and Explaining how Concepts are Encoded by Filters in Deep Neural Networks](https://mlsanity.com/paper/1801.03454/) (2018)
- [Hypercolumns for Object Segmentation and Fine-grained Localization](https://mlsanity.com/paper/1411.5752/) (2014)
- [What shapes feature representations? Exploring datasets, architectures, and training](https://mlsanity.com/paper/2006.12433/) (2020)
- [A Disentangling Invertible Interpretation Network for Explaining Latent Representations](https://mlsanity.com/paper/2004.13166/) (2020)
- [The Origins and Prevalence of Texture Bias in Convolutional Neural Networks](https://mlsanity.com/paper/1911.09071/) (2019)

## Built on by (most cited first, 30 of 52)

- [Intriguing Properties of Vision Transformers](https://mlsanity.com/paper/2105.10497/) (2021)
- [Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations](https://mlsanity.com/paper/2204.02937/) (2022)
- [Weakly-Supervised Multi-Granularity Map Learning for Vision-and-Language Navigation](https://mlsanity.com/paper/2210.07506/) (2022)
- [Mechanistic Mode Connectivity](https://mlsanity.com/paper/2211.08422/) (2022)
- [Robust Contrastive Learning Using Negative Samples with Diminished Semantics](https://mlsanity.com/paper/2110.14189/) (2021)
- [Enhancing Self-supervised Video Representation Learning via Multi-level Feature Optimization](https://mlsanity.com/paper/2108.02183/) (2021)
- [Position, Padding and Predictions: A Deeper Look at Position Information in CNNs](https://mlsanity.com/paper/2101.12322/) (2021)
- [A Deeper Dive Into What Deep Spatiotemporal Networks Encode: Quantifying Static vs. Dynamic Information](https://mlsanity.com/paper/2206.02846/) (2022)
- [Large Separable Kernel Attention: Rethinking the Large Kernel Attention Design in CNN](https://mlsanity.com/paper/2309.01439/) (2023)
- [Deep Digging into the Generalization of Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2205.11083/) (2022)
- [Texture Learning Domain Randomization for Domain Generalized Segmentation](https://mlsanity.com/paper/2303.11546/) (2023)
- [Can We Talk Models Into Seeing the World Differently?](https://mlsanity.com/paper/2403.09193/) (2024)
- [FGPrompt: Fine-grained Goal Prompting for Image-goal Navigation](https://mlsanity.com/paper/2310.07473/) (2023)
- [Can Biases in ImageNet Models Explain Generalization?](https://mlsanity.com/paper/2404.01509/) (2024)
- [A Game-Theoretic Taxonomy of Visual Concepts in DNNs](https://mlsanity.com/paper/2106.10938/) (2021)
- [ImageNet-trained CNNs are not biased towards texture: Revisiting feature reliance through controlled suppression](https://mlsanity.com/paper/2509.20234/) (2025)
- [Global Pooling, More than Meets the Eye: Position Information is Encoded Channel-Wise in CNNs](https://mlsanity.com/paper/2108.07884/) (2021)
- [Convolutional Neural Networks Rarely Learn Shape for Semantic Segmentation](https://mlsanity.com/paper/2305.06568/) (2023)
- [Trapped in texture bias? A large scale comparison of deep instance segmentation](https://mlsanity.com/paper/2401.09109/) (2024)
- [SegMix: Co-occurrence Driven Mixup for Semantic Segmentation and Adversarial Robustness](https://mlsanity.com/paper/2108.09929/) (2021)
- [Quantifying and Learning Static vs. Dynamic Information in Deep Spatiotemporal Networks](https://mlsanity.com/paper/2211.01783/) (2022)
- [Exploiting Shape Cues for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2208.04286/) (2022)
- [CLAD: A Contrastive Learning based Approach for Background Debiasing](https://mlsanity.com/paper/2210.02748/) (2022)
- [Deep Shape-Texture Statistics for Completely Blind Image Quality Evaluation](https://mlsanity.com/paper/2401.08107/) (2024)
- [Graph Jigsaw Learning for Cartoon Face Recognition](https://mlsanity.com/paper/2107.06532/) (2021)
- [A Study on the Generality of Neural Network Structures for Monocular Depth Estimation](https://mlsanity.com/paper/2301.03169/) (2023)
- [Unveiling the Tapestry: the Interplay of Generalization and Forgetting in Continual Learning](https://mlsanity.com/paper/2211.11174/) (2022)
- [SETA: Semantic-Aware Token Augmentation for Domain Generalization](https://mlsanity.com/paper/2403.11792/) (2024)
- [Shape Bias and Robustness Evaluation via Cue Decomposition for Image Classification and Segmentation](https://mlsanity.com/paper/2503.12453/) (2025)
- [AI-Based Detection, Classification and Prediction/Prognosis in Medical Imaging: Towards Radiophenomics](https://mlsanity.com/paper/2110.10332/) (2021)

## Abstract (arXiv)

Contrasting the previous evidence that neurons in the later layers of a Convolutional Neural Network (CNN) respond to complex object shapes, recent studies have shown that CNNs actually exhibit a `texture bias': given an image with both texture and shape cues (e.g., a stylized image), a CNN is biased towards predicting the category corresponding to the texture. However, these previous studies conduct experiments on the final classification output of the network, and fail to robustly evaluate the bias contained (i) in the latent representations, and (ii) on a per-pixel level. In this paper, we design a series of experiments that overcome these issues. We do this with the goal of better understanding what type of shape information contained in the network is discriminative, where shape information is encoded, as well as when the network learns about object shape during training. We show that a network learns the majority of overall shape information at the first few epochs of training and that this information is largely encoded in the last few layers of a CNN. Finally, we show that the encoding of shape does not imply the encoding of localized per-pixel semantic information. The experimental results and findings provide a more accurate understanding of the behaviour of current CNNs, thus helping to inform future design choices.