# When do GANs replicate? On the choice of dataset size

arXiv: https://arxiv.org/abs/2202.11765 · HTML: https://mlsanity.com/paper/2202.11765/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 30 references that are in the map.
- First posted on arXiv 23 Feb 2022.

## Builds on (most cited first, 20 of 20)

- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Assessing Generative Models via Precision and Recall](https://mlsanity.com/paper/1806.00035/) (2018)
- [A note on the evaluation of generative models](https://mlsanity.com/paper/1511.01844/) (2015)
- [Are GANs Created Equal? A Large-Scale Study](https://mlsanity.com/paper/1711.10337/) (2017)
- [GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification](https://mlsanity.com/paper/1803.01229/) (2018)
- [An empirical study on evaluation metrics of generative adversarial networks](https://mlsanity.com/paper/1806.07755/) (2018)
- [On the Intrinsic Dimensionality of Image Representations](https://mlsanity.com/paper/1803.09672/) (2018)
- [Computing the Testing Error without a Testing Set](https://mlsanity.com/paper/2005.00450/) (2020)
- [Detecting Overfitting of Deep Generative Networks via Latent Recovery](https://mlsanity.com/paper/1901.03396/) (2019)
- [This Face Does Not Exist ... But It Might Be Yours! Identity Leakage in Generative Models](https://mlsanity.com/paper/2101.05084/) (2020)
- [Do GANs actually learn the distribution? An empirical study](https://mlsanity.com/paper/1706.08224/) (2017)
- [Deepfakes for Medical Video De-Identification: Privacy Protection and Diagnostic Information Preservation](https://mlsanity.com/paper/2003.00813/) (2020)
- [Empirical Analysis of Overfitting and Mode Drop in GAN Training](https://mlsanity.com/paper/2006.14265/) (2020)

## Built on by (most cited first, 30 of 34)

- [Extracting Training Data from Diffusion Models](https://mlsanity.com/paper/2301.13188/) (2023)
- [Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models](https://mlsanity.com/paper/2212.03860/) (2022)
- [On Memorization in Diffusion Models](https://mlsanity.com/paper/2310.02664/) (2023)
- [Machine Learning for Synthetic Data Generation: A Review](https://mlsanity.com/paper/2302.04062/) (2023)
- [Erasing Undesirable Influence in Diffusion Models](https://mlsanity.com/paper/2401.05779/) (2024)
- [A Closer Look at Few-shot Image Generation](https://mlsanity.com/paper/2205.03805/) (2022)
- [Analyzing Bias in Diffusion-based Face Generation Models](https://mlsanity.com/paper/2305.06402/) (2023)
- [The Journey, Not the Destination: How Data Guides Diffusion Models](https://mlsanity.com/paper/2312.06205/) (2023)
- [A Geometric Framework for Understanding Memorization in Generative Models](https://mlsanity.com/paper/2411.00113/) (2024)
- [A Pathway Towards Responsible AI Generated Content](https://mlsanity.com/paper/2303.01325/) (2023)
- [Few-shot Image Generation via Adaptation-Aware Kernel Modulation](https://mlsanity.com/paper/2210.16559/) (2022)
- [Synthetic Data in Human Analysis: A Survey](https://mlsanity.com/paper/2208.09191/) (2022)
- [One Model to Unite Them All: Personalized Federated Learning of Multi-Contrast MRI Synthesis](https://mlsanity.com/paper/2207.06509/) (2022)
- [Few-shot Image Generation via Masked Discrimination](https://mlsanity.com/paper/2210.15194/) (2022)
- [Frame by Familiar Frame: Understanding Replication in Video Diffusion Models](https://mlsanity.com/paper/2403.19593/) (2024)
- [Haven't I Seen You Before? Assessing Identity Leakage in Synthetic Irises](https://mlsanity.com/paper/2211.05629/) (2022)
- [An Inversion-based Measure of Memorization for Diffusion Models](https://mlsanity.com/paper/2405.05846/) (2024)
- [Trustworthy Machine Learning via Memorization and the Granular Long-Tail: A Survey on Interactions, Tradeoffs, and Beyond](https://mlsanity.com/paper/2503.07501/) (2025)
- [Generative Autoregressive Transformers for Model-Agnostic Federated MRI Reconstruction](https://mlsanity.com/paper/2502.04521/) (2025)
- [Reducing Training Sample Memorization in GANs by Training with Memorization Rejection](https://mlsanity.com/paper/2210.12231/) (2022)
- [MedAugment: Universal Automatic Data Augmentation Plug-in for Medical Image Analysis](https://mlsanity.com/paper/2306.17466/) (2023)
- [Segmenting Medical Images with Limited Data](https://mlsanity.com/paper/2407.09189/) (2024)
- [Generalization in VAE and Diffusion Models: A Unified Information-Theoretic Analysis](https://mlsanity.com/paper/2506.00849/) (2025)
- [Hyperparameter-Free Medical Image Synthesis for Sharing Data and Improving Site-Specific Segmentation](https://mlsanity.com/paper/2404.06240/) (2024)
- [Fairer Analysis and Demographically Balanced Face Generation for Fairer Face Verification](https://mlsanity.com/paper/2412.03349/) (2024)
- [PetroGAN: A novel GAN-based approach to generate realistic, label-free petrographic datasets](https://mlsanity.com/paper/2204.05114/) (2022)
- [On the causality-preservation capabilities of generative modelling](https://mlsanity.com/paper/2301.01109/) (2023)
- [AdAM: Few-Shot Image Generation via Adaptation-Aware Kernel Modulation](https://mlsanity.com/paper/2307.01465/) (2023)
- [LatentKeypointGAN: Controlling Images via Latent Keypoints -- Extended Abstract](https://mlsanity.com/paper/2205.03448/) (2022)
- [SFHarmony: Source Free Domain Adaptation for Distributed Neuroimaging Analysis](https://mlsanity.com/paper/2303.15965/) (2023)

## Abstract (arXiv)

Do GANs replicate training images? Previous studies have shown that GANs do not seem to replicate training data without significant change in the training procedure. This leads to a series of research on the exact condition needed for GANs to overfit to the training data. Although a number of factors has been theoretically or empirically identified, the effect of dataset size and complexity on GANs replication is still unknown. With empirical evidence from BigGAN and StyleGAN2, on datasets CelebA, Flower and LSUN-bedroom, we show that dataset size and its complexity play an important role in GANs replication and perceptual quality of the generated images. We further quantify this relationship, discovering that replication percentage decays exponentially with respect to dataset size and complexity, with a shared decaying factor across GAN-dataset combinations. Meanwhile, the perceptual image quality follows a U-shape trend w.r.t dataset size. This finding leads to a practical tool for one-shot estimation on minimal dataset size to prevent GAN replication which can be used to guide datasets construction and selection.