# A Non-Parametric Test to Detect Data-Copying in Generative Models

arXiv: https://arxiv.org/abs/2004.05675 · HTML: https://mlsanity.com/paper/2004.05675/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Apr 2020.

## Built on by (most cited first, 30 of 52)

- [Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus](https://mlsanity.com/paper/2104.08758/) (2021)
- [Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models](https://mlsanity.com/paper/2212.03860/) (2022)
- [Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models](https://mlsanity.com/paper/2306.04675/) (2023)
- [How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Models](https://mlsanity.com/paper/2102.08921/) (2021)
- [Pros and Cons of GAN Evaluation Measures: New Developments](https://mlsanity.com/paper/2103.09396/) (2021)
- [Machine Learning for Synthetic Data Generation: A Review](https://mlsanity.com/paper/2302.04062/) (2023)
- [REaLTabFormer: Generating Realistic Relational and Tabular Data using Transformers](https://mlsanity.com/paper/2302.02041/) (2023)
- [Do Language Models Plagiarize?](https://mlsanity.com/paper/2203.07618/) (2022)
- [On Memorization in Probabilistic Deep Generative Models](https://mlsanity.com/paper/2106.03216/) (2021)
- [Erasing Undesirable Influence in Diffusion Models](https://mlsanity.com/paper/2401.05779/) (2024)
- [A Geometric Framework for Understanding Memorization in Generative Models](https://mlsanity.com/paper/2411.00113/) (2024)
- [A Pathway Towards Responsible AI Generated Content](https://mlsanity.com/paper/2303.01325/) (2023)
- [DiffInfinite: Large Mask-Image Synthesis via Parallel Random Patch Diffusion in Histopathology](https://mlsanity.com/paper/2306.13384/) (2023)
- [Feature Likelihood Divergence: Evaluating the Generalization of Generative Models Using Samples](https://mlsanity.com/paper/2302.04440/) (2023)
- [Cousins Of The Vendi Score: A Family Of Similarity-Based Diversity Metrics For Science And Machine Learning](https://mlsanity.com/paper/2310.12952/) (2023)
- [Understanding Instance-based Interpretability of Variational Auto-Encoders](https://mlsanity.com/paper/2105.14203/) (2021)
- [On Training Sample Memorization: Lessons from Benchmarking Generative Modeling with a Large-scale Competition](https://mlsanity.com/paper/2106.03062/) (2021)
- [An Interpretable Evaluation of Entropy-based Novelty of Generative Models](https://mlsanity.com/paper/2402.17287/) (2024)
- [Unveiling Differences in Generative Models: A Scalable Differential Clustering Approach](https://mlsanity.com/paper/2405.02700/) (2024)
- [Generative Modeling of Weights: Generalization or Memorization?](https://mlsanity.com/paper/2506.07998/) (2025)
- [Metrics that matter: Evaluating image quality metrics for medical image generation](https://mlsanity.com/paper/2505.07175/) (2025)
- [Frame by Familiar Frame: Understanding Replication in Video Diffusion Models](https://mlsanity.com/paper/2403.19593/) (2024)
- [Forte : Finding Outliers with Representation Typicality Estimation](https://mlsanity.com/paper/2410.01322/) (2024)
- [Generalization Metrics for Practical Quantum Advantage in Generative Models](https://mlsanity.com/paper/2201.08770/) (2022)
- [DP-SGD vs PATE: Which Has Less Disparate Impact on GANs?](https://mlsanity.com/paper/2111.13617/) (2021)
- [The Vendiscope: An Algorithmic Microscope For Data Collections](https://mlsanity.com/paper/2502.10828/) (2025)
- [Reducing Training Sample Memorization in GANs by Training with Memorization Rejection](https://mlsanity.com/paper/2210.12231/) (2022)
- [Data Pruning in Generative Diffusion Models](https://mlsanity.com/paper/2411.12523/) (2024)
- [GReaTER: Generate Realistic Tabular data after data Enhancement and Reduction](https://mlsanity.com/paper/2503.15564/) (2025)
- [Generalization in VAE and Diffusion Models: A Unified Information-Theoretic Analysis](https://mlsanity.com/paper/2506.00849/) (2025)

## Abstract (arXiv)

Detecting overfitting in generative models is an important challenge in machine learning. In this work, we formalize a form of overfitting that we call emdata-copying -- where the generative model memorizes and outputs training samples or small variations thereof. We provide a three sample non-parametric test for detecting data-copying that uses the training set, a separate sample from the target distribution, and a generated sample from the model, and study the performance of our test on several canonical models and datasets. For code & examples, visit https://github.com/casey-meehan/data-copying