# A Data-Centric Approach for Training Deep Neural Networks with Less Data

arXiv: https://arxiv.org/abs/2110.03613 · HTML: https://mlsanity.com/paper/2110.03613/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 6 references that are in the map.
- First posted on arXiv 7 Oct 2021.

## Builds on (most cited first, 3 of 3)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)

## Built on by (most cited first, 28 of 28)

- [Teaching Arithmetic to Small Transformers](https://mlsanity.com/paper/2307.03381/) (2023)
- [Transformers Can Achieve Length Generalization But Not Robustly](https://mlsanity.com/paper/2402.09371/) (2024)
- [Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-aligned Quality Estimation](https://mlsanity.com/paper/2402.18191/) (2024)
- [Is one annotation enough? A data-centric image classification benchmark for noisy and ambiguous label estimation](https://mlsanity.com/paper/2207.06214/) (2022)
- [VariErr NLI: Separating Annotation Error from Human Label Variation](https://mlsanity.com/paper/2403.01931/) (2024)
- [The Principles of Data-Centric AI (DCAI)](https://mlsanity.com/paper/2211.14611/) (2022)
- [AlpaGasus: Training A Better Alpaca with Fewer Data](https://mlsanity.com/paper/2307.08701/) (2023)
- [Measuring Data](https://mlsanity.com/paper/2212.05129/) (2022)
- [Data-Driven Design for Metamaterials and Multiscale Systems: A Review](https://mlsanity.com/paper/2307.05506/) (2023)
- [Unsupervised classification to improve the quality of a bird song recording dataset](https://mlsanity.com/paper/2302.07560/) (2023)
- [A Data-Centric Solution to NonHomogeneous Dehazing via Vision Transformer](https://mlsanity.com/paper/2304.07874/) (2023)
- [GDeR: Safeguarding Efficiency, Balancing, and Robustness via Prototypical Graph Pruning](https://mlsanity.com/paper/2410.13761/) (2024)
- [Can Synthetic Audio From Generative Foundation Models Assist Audio Recognition and Speech Modeling?](https://mlsanity.com/paper/2406.08800/) (2024)
- [General Information Metrics for Improving AI Model Training Efficiency](https://mlsanity.com/paper/2501.02004/) (2025)
- [Label Smarter, Not Harder: CleverLabel for Faster Annotation of Ambiguous Image Classification with Higher Quality](https://mlsanity.com/paper/2305.12811/) (2023)
- [Provable advantages of kernel-based quantum learners and quantum preprocessing based on Grover's algorithm](https://mlsanity.com/paper/2309.14406/) (2023)
- [POP: Mining POtential Performance of new fashion products via webly cross-modal query expansion](https://mlsanity.com/paper/2207.11001/) (2022)
- [Data-Centric Approach to Constrained Machine Learning: A Case Study on Conway's Game of Life](https://mlsanity.com/paper/2408.12778/) (2024)
- [Smart Active Sampling to enhance Quality Assurance Efficiency](https://mlsanity.com/paper/2209.11464/) (2022)
- [Thresholding Data Shapley for Data Cleansing Using Multi-Armed Bandits](https://mlsanity.com/paper/2402.08209/) (2024)
- [Maintaining Performance with Less Data](https://mlsanity.com/paper/2208.02007/) (2022)
- [Deep Learning based pipeline for anomaly detection and quality enhancement in industrial binder jetting processes](https://mlsanity.com/paper/2209.10178/) (2022)
- [Synfeal: A Data-Driven Simulator for End-to-End Camera Localization](https://mlsanity.com/paper/2305.18260/) (2023)
- [Class-Adaptive Sampling Policy for Efficient Continual Learning](https://mlsanity.com/paper/2311.16485/) (2023)
- [CUDC: A Curiosity-Driven Unsupervised Data Collection Method with Adaptive Temporal Distances for Offline Reinforcement Learning](https://mlsanity.com/paper/2312.12191/) (2023)
- [Continual Learning: Less Forgetting, More OOD Generalization via Adaptive Contrastive Replay](https://mlsanity.com/paper/2410.07110/) (2024)
- [Breaking the Statistical Similarity Trap in Extreme Convection Detection](https://mlsanity.com/paper/2509.09195/) (2025)
- [A Hierarchical Consistency Framework for Auditing Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2609.07075/) (2026)

## Abstract (arXiv)

While the availability of large datasets is perceived to be a key requirement for training deep neural networks, it is possible to train such models with relatively little data. However, compensating for the absence of large datasets demands a series of actions to enhance the quality of the existing samples and to generate new ones. This paper summarizes our winning submission to the "Data-Centric AI" competition. We discuss some of the challenges that arise while training with a small dataset, offer a principled approach for systematic data quality enhancement, and propose a GAN-based solution for synthesizing new data points. Our evaluations indicate that the dataset generated by the proposed pipeline offers 5% accuracy improvement while being significantly smaller than the baseline.