# CLIP: Train Faster with Less Data

arXiv: https://arxiv.org/abs/2212.01452 · HTML: https://mlsanity.com/paper/2212.01452/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 43 references that are in the map.
- First posted on arXiv 2 Dec 2022.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Deep Learning on a Data Diet: Finding Important Examples Early in Training](https://mlsanity.com/paper/2107.07075/) (2021)
- [Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey](https://mlsanity.com/paper/2003.04960/) (2020)
- [On The Power of Curriculum Learning in Training Deep Networks](https://mlsanity.com/paper/1904.03626/) (2019)
- [CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenes](https://mlsanity.com/paper/1802.10062/) (2018)
- [Competence-based Curriculum Learning for Neural Machine Translation](https://mlsanity.com/paper/1903.09848/) (2019)
- [Drone-based Object Counting by Spatially Regularized Regional Proposal Network](https://mlsanity.com/paper/1707.05972/) (2017)
- [Context-Aware Crowd Counting](https://mlsanity.com/paper/1811.10452/) (2018)
- [Dataset Pruning: Reducing Training Data by Examining Generalization Influence](https://mlsanity.com/paper/2205.09329/) (2022)
- [When Do Curricula Work?](https://mlsanity.com/paper/2012.03107/) (2020)
- [Switching Convolutional Neural Network for Crowd Counting](https://mlsanity.com/paper/1708.00199/) (2017)
- [Crowd Counting and Density Estimation by Trellis Encoder-Decoder Network](https://mlsanity.com/paper/1903.00853/) (2019)
- [CrowdNet: A Deep Convolutional Network for Dense Crowd Counting](https://mlsanity.com/paper/1608.06197/) (2016)
- [CNN-based Cascaded Multi-task Learning of High-level Prior and Density Estimation for Crowd Counting](https://mlsanity.com/paper/1707.09605/) (2017)
- [Attention-Guided Curriculum Learning for Weakly Supervised Classification and Localization of Thoracic Diseases on Chest Radiographs](https://mlsanity.com/paper/1807.07532/) (2018)
- [Multi-scale Convolutional Neural Networks for Crowd Counting](https://mlsanity.com/paper/1702.02359/) (2017)
- [Self Paced Deep Learning for Weakly Supervised Object Detection](https://mlsanity.com/paper/1605.07651/) (2016)
- [Dynamically Composing Domain-Data Selection with Clean-Data Selection by "Co-Curricular Learning" for Neural Machine Translation](https://mlsanity.com/paper/1906.01130/) (2019)
- [Revisiting Crowd Counting: State-of-the-art, Trends, and Future Perspectives](https://mlsanity.com/paper/2209.07271/) (2022)
- [TAFNet: A Three-Stream Adaptive Fusion Network for RGB-T Crowd Counting](https://mlsanity.com/paper/2202.08517/) (2022)
- [Global Sum Pooling: A Generalization Trick for Object Counting with Small Datasets of Large Images](https://mlsanity.com/paper/1805.11123/) (2018)
- [A Unified Multi-Task Learning Framework of Real-Time Drone Supervision for Crowd Counting](https://mlsanity.com/paper/2202.03843/) (2022)

## Built on by (most cited first, 4 of 4)

- [Data Pruning via Separability, Integrity, and Model Uncertainty-Aware Importance Sampling](https://mlsanity.com/paper/2409.13915/) (2024)
- [Curriculum for Crowd Counting -- Is it Worthy?](https://mlsanity.com/paper/2401.07586/) (2024)
- [Breaking the Limits of Open-Weight CLIP: An Optimization Framework for Self-supervised Fine-tuning of CLIP](https://mlsanity.com/paper/2601.09859/) (2026)
- [Accelerating Deep Learning with Fixed Time Budget](https://mlsanity.com/paper/2410.03790/) (2024)

## Abstract (arXiv)

Deep learning models require an enormous amount of data for training. However, recently there is a shift in machine learning from model-centric to data-centric approaches. In data-centric approaches, the focus is to refine and improve the quality of the data to improve the learning performance of the models rather than redesigning model architectures. In this paper, we propose CLIP i.e., Curriculum Learning with Iterative data Pruning. CLIP combines two data-centric approaches i.e., curriculum learning and dataset pruning to improve the model learning accuracy and convergence speed. The proposed scheme applies loss-aware dataset pruning to iteratively remove the least significant samples and progressively reduces the size of the effective dataset in the curriculum learning training. Extensive experiments performed on crowd density estimation models validate the notion behind combining the two approaches by reducing the convergence time and improving generalization. To our knowledge, the idea of data pruning as an embedded process in curriculum learning is novel.