# Convolutional Neural Networks Rarely Learn Shape for Semantic Segmentation

arXiv: https://arxiv.org/abs/2305.06568 · HTML: https://mlsanity.com/paper/2305.06568/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 60 references that are in the map.
- First posted on arXiv 11 May 2023.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Certified Adversarial Robustness via Randomized Smoothing](https://mlsanity.com/paper/1902.02918/) (2019)
- [Adversarial Examples Are Not Bugs, They Are Features](https://mlsanity.com/paper/1905.02175/) (2019)
- [Ensemble Adversarial Training: Attacks and Defenses](https://mlsanity.com/paper/1705.07204/) (2017)
- [Adversarial Training for Free!](https://mlsanity.com/paper/1904.12843/) (2019)
- [Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNs](https://mlsanity.com/paper/2203.06717/) (2022)
- [On Feature Learning in the Presence of Spurious Correlations](https://mlsanity.com/paper/2210.11369/) (2022)
- [What shapes feature representations? Exploring datasets, architectures, and training](https://mlsanity.com/paper/2006.12433/) (2020)
- [More Diverse Means Better: Multimodal Deep Learning Meets Remote Sensing Imagery Classification](https://mlsanity.com/paper/2008.05457/) (2020)
- [Controlling Perceptual Factors in Neural Style Transfer](https://mlsanity.com/paper/1611.07865/) (2016)
- [Association of genomic subtypes of lower-grade gliomas with shape features automatically extracted by a deep learning algorithm](https://mlsanity.com/paper/1906.03720/) (2019)
- [Shape-Texture Debiased Neural Network Training](https://mlsanity.com/paper/2010.05981/) (2020)
- [An Augmented Linear Mixing Model to Address Spectral Variability for Hyperspectral Unmixing](https://mlsanity.com/paper/1810.12000/) (2018)
- [Shape or Texture: Understanding Discriminative Features in CNNs](https://mlsanity.com/paper/2101.11604/) (2021)
- [Cognitive Psychology for Deep Neural Networks: A Shape Bias Case Study](https://mlsanity.com/paper/1706.08606/) (2017)
- [Does enhanced shape bias improve neural network robustness to common corruptions?](https://mlsanity.com/paper/2104.09789/) (2021)
- [Assessing Shape Bias Property of Convolutional Neural Networks](https://mlsanity.com/paper/1803.07739/) (2018)
- [Detection of masses and architectural distortions in digital breast tomosynthesis: a publicly available dataset of 5,060 patients and a deep learning model](https://mlsanity.com/paper/2011.07995/) (2020)

## Built on by (most cited first, 12 of 12)

- [Quantifying the Limits of Segmentation Foundation Models: Modeling Challenges in Segmenting Tree-Like and Low-Contrast Objects](https://mlsanity.com/paper/2412.04243/) (2024)
- [Automated Muscle and Fat Segmentation in Computed Tomography for Comprehensive Body Composition Analysis](https://mlsanity.com/paper/2502.09779/) (2025)
- [LGD: Leveraging Generative Descriptions for Zero-Shot Referring Image Segmentation](https://mlsanity.com/paper/2504.14467/) (2025)
- [Decomposing and Revising What Language Models Generate](https://mlsanity.com/paper/2509.00765/) (2025)
- [LERENet: Eliminating Intra-class Differences for Metal Surface Defect Few-shot Semantic Segmentation](https://mlsanity.com/paper/2403.11122/) (2024)
- [Rethinking Pulmonary Embolism Segmentation: A Study of Current Approaches and Challenges with an Open Weight Model](https://mlsanity.com/paper/2509.18308/) (2025)
- [On the Influence of Shape, Texture and Color for Learning Semantic Segmentation](https://mlsanity.com/paper/2410.14878/) (2024)
- [Contrast All the Time: Learning Time Series Representation from Temporal Consistency](https://mlsanity.com/paper/2410.15416/) (2024)
- [Toward Semantic-Agnostic and Shape-Aware Vision-Language Segmentation Models](https://mlsanity.com/paper/2605.28348/) (2026)
- [Investigating the Visual Cues of CNNs for Vascular Segmentation: A Case Study in Microscopy and Fundus Imaging](https://mlsanity.com/paper/2607.23371/) (2026)
- [Shape-Based Inductive Bias for Glioma Grading from Tumor Contours](https://mlsanity.com/paper/2607.26090/) (2026)
- [Temporally Ordered Region-Token Mamba with Logit-Space Diffusion for Remote Sensing Change Detection](https://mlsanity.com/paper/2609.27149/) (2026)

## Abstract (arXiv)

Shape learning, or the ability to leverage shape information, could be a desirable property of convolutional neural networks (CNNs) when target objects have specific shapes. While some research on the topic is emerging, there is no systematic study to conclusively determine whether and under what circumstances CNNs learn shape. Here, we present such a study in the context of segmentation networks where shapes are particularly important. We define shape and propose a new behavioral metric to measure the extent to which a CNN utilizes shape information. We then execute a set of experiments with synthetic and real-world data to progressively uncover under which circumstances CNNs learn shape and what can be done to encourage such behavior. We conclude that (i) CNNs do not learn shape in typical settings but rather rely on other features available to identify the objects of interest, (ii) CNNs can learn shape, but only if the shape is the only feature available to identify the object, (iii) sufficiently large receptive field size relative to the size of target objects is necessary for shape learning; (iv) a limited set of augmentations can encourage shape learning; (v) learning shape is indeed useful in the presence of out-of-distribution data.