# Impact of Data Duplication on Deep Neural Network-Based Image Classifiers: Robust vs. Standard Models

arXiv: https://arxiv.org/abs/2504.00638 · HTML: https://mlsanity.com/paper/2504.00638/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 25 references that are in the map.
- First posted on arXiv 1 Apr 2025.

## Builds on (most cited first, 12 of 12)

- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Adversarial Examples Are Not Bugs, They Are Features](https://mlsanity.com/paper/1905.02175/) (2019)
- [Quantifying Memorization Across Neural Language Models](https://mlsanity.com/paper/2202.07646/) (2022)
- [Deduplicating Training Data Makes Language Models Better](https://mlsanity.com/paper/2107.06499/) (2021)
- [What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation](https://mlsanity.com/paper/2008.03703/) (2020)
- [Adversarial Training and Robustness for Multiple Perturbations](https://mlsanity.com/paper/1904.13000/) (2019)
- [Do We Train on Test Data? Purging CIFAR of Near-Duplicates](https://mlsanity.com/paper/1902.00423/) (2019)
- [It Is All About Data: A Survey on the Effects of Data on Adversarial Robustness](https://mlsanity.com/paper/2303.09767/) (2023)
- [CE-Dedup: Cost-Effective Convolutional Neural Nets Training based on Image Deduplication](https://mlsanity.com/paper/2109.00899/) (2021)
- [Vulnerability Under Adversarial Machine Learning: Bias or Variance?](https://mlsanity.com/paper/2008.00138/) (2020)
- [Privacy-Preserving Data Deduplication for Enhancing Federated Learning of Language Models (Extended Version)](https://mlsanity.com/paper/2407.08152/) (2024)

## Built on by (most cited first, 4 of 4)

- [The Landscape of Memorization in LLMs: Mechanisms, Measurement, and Mitigation](https://mlsanity.com/paper/2507.05578/) (2025)
- [ContextShift: A Controlled Benchmark for Context Dependence in Object Detection](https://mlsanity.com/paper/2606.09495/) (2026)
- [Limitations of Synthetic Data Generation in Specialized Data-Scarce Domains](https://mlsanity.com/paper/2608.13729/) (2026)
- [I Have a Stream: Making Self-Supervised Learning Work on Continuous Video](https://mlsanity.com/paper/2609.40333/) (2026)

## Abstract (arXiv)

The accuracy and robustness of machine learning models against adversarial attacks are significantly influenced by factors such as training data quality, model architecture, the training process, and the deployment environment. In recent years, duplicated data in training sets, especially in language models, has attracted considerable attention. It has been shown that deduplication enhances both training performance and model accuracy in language models. While the importance of data quality in training image classifier Deep Neural Networks (DNNs) is widely recognized, the impact of duplicated images in the training set on model generalization and performance has received little attention. In this paper, we address this gap and provide a comprehensive study on the effect of duplicates in image classification. Our analysis indicates that the presence of duplicated images in the training set not only negatively affects the efficiency of model training but also may result in lower accuracy of the image classifier. This negative impact of duplication on accuracy is particularly evident when duplicated data is non-uniform across classes or when duplication, whether uniform or non-uniform, occurs in the training set of an adversarially trained model. Even when duplicated samples are selected in a uniform way, increasing the amount of duplication does not lead to a significant improvement in accuracy.