# Harnessing Synthetic Datasets: The Role of Shape Bias in Deep Neural Network Generalization

arXiv: https://arxiv.org/abs/2311.06224 · HTML: https://mlsanity.com/paper/2311.06224/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 36 references that are in the map.
- First posted on arXiv 10 Nov 2023.

## Builds on (most cited first, 24 of 24)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Scaling Vision Transformers to 22 Billion Parameters](https://mlsanity.com/paper/2302.05442/) (2023)
- [DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effort](https://mlsanity.com/paper/2104.06490/) (2021)
- [Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet](https://mlsanity.com/paper/1904.00760/) (2019)
- [Towards Understanding the Spectral Bias of Deep Learning](https://mlsanity.com/paper/1912.01198/) (2019)
- [Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization](https://mlsanity.com/paper/2104.05833/) (2021)
- [Classification Accuracy Score for Conditional Generative Models](https://mlsanity.com/paper/1905.10887/) (2019)
- [Generative Models as a Data Source for Multiview Representation Learning](https://mlsanity.com/paper/2106.05258/) (2021)
- [Pre-training without Natural Images](https://mlsanity.com/paper/2101.08515/) (2021)
- [Learning to See by Looking at Noise](https://mlsanity.com/paper/2106.05963/) (2021)
- [Repurposing GANs for One-shot Semantic Part Segmentation](https://mlsanity.com/paper/2103.04379/) (2021)
- [Measuring the tendency of CNNs to Learn Surface Statistical Regularities](https://mlsanity.com/paper/1711.11561/) (2017)
- [Shape-Texture Debiased Neural Network Training](https://mlsanity.com/paper/2010.05981/) (2020)
- [This dataset does not exist: training models from generated images](https://mlsanity.com/paper/1911.02888/) (2019)
- [Inductive Bias of Deep Convolutional Networks through Pooling Geometry](https://mlsanity.com/paper/1605.06743/) (2016)
- [A Theoretical Study of Inductive Biases in Contrastive Learning](https://mlsanity.com/paper/2211.14699/) (2022)
- [Finding the Needle in the Haystack with Convolutions: on the benefits of architectural bias](https://mlsanity.com/paper/1906.06766/) (2019)
- [A Random CNN Sees Objects: One Inductive Bias of CNN and Its Applications](https://mlsanity.com/paper/2106.09259/) (2021)
- [Learning Inductive Biases with Simple Neural Networks](https://mlsanity.com/paper/1802.02745/) (2018)
- [The Origins and Prevalence of Texture Bias in Convolutional Neural Networks](https://mlsanity.com/paper/1911.09071/) (2019)

## Built on by (most cited first, 4 of 4)

- [Is Synthetic Data all We Need? Benchmarking the Robustness of Models Trained with Synthetic Images](https://mlsanity.com/paper/2405.20469/) (2024)
- [Can We Talk Models Into Seeing the World Differently?](https://mlsanity.com/paper/2403.09193/) (2024)
- [SPIE: Semantic and Structural Post-Training of Image Editing Diffusion Models with AI feedback](https://mlsanity.com/paper/2504.12833/) (2025)
- [Hidden in Plain Sight: Evaluating Abstract Shape Recognition in Vision-Language Models](https://mlsanity.com/paper/2411.06287/) (2024)

## Abstract (arXiv)

Recent advancements in deep learning have been primarily driven by the use of large models trained on increasingly vast datasets. While neural scaling laws have emerged to predict network performance given a specific level of computational resources, the growing demand for expansive datasets raises concerns. To address this, a new research direction has emerged, focusing on the creation of synthetic data as a substitute. In this study, we investigate how neural networks exhibit shape bias during training on synthetic datasets, serving as an indicator of the synthetic data quality. Specifically, our findings indicate three key points: (1) Shape bias varies across network architectures and types of supervision, casting doubt on its reliability as a predictor for generalization and its ability to explain differences in model recognition compared to human capabilities. (2) Relying solely on shape bias to estimate generalization is unreliable, as it is entangled with diversity and naturalism. (3) We propose a novel interpretation of shape bias as a tool for estimating the diversity of samples within a dataset. Our research aims to clarify the implications of using synthetic data and its associated shape bias in deep learning, addressing concerns regarding generalization and dataset quality.