# Are all training examples equally valuable?

arXiv: https://arxiv.org/abs/1311.6510 · HTML: https://mlsanity.com/paper/1311.6510/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Nov 2013.

## Built on by (most cited first, 14 of 14)

- [Dataset Distillation by Matching Training Trajectories](https://mlsanity.com/paper/2203.11932/) (2022)
- [DREAM: Efficient Dataset Distillation by Representative Matching](https://mlsanity.com/paper/2302.14416/) (2023)
- [Compressed Gastric Image Generation Based on Soft-Label Dataset Distillation for Medical Data Sharing](https://mlsanity.com/paper/2209.14635/) (2022)
- [Iterative Teaching by Data Hallucination](https://mlsanity.com/paper/2210.17467/) (2022)
- [DREAM+: Efficient Dataset Distillation by Bidirectional Representative Matching](https://mlsanity.com/paper/2310.15052/) (2023)
- [A Survey of Dataset Refinement for Problems in Computer Vision Datasets](https://mlsanity.com/paper/2210.11717/) (2022)
- [Learning to Combat Noisy Labels via Classification Margins](https://mlsanity.com/paper/2102.00751/) (2021)
- [AST: Effective Dataset Distillation through Alignment with Smooth and High-Quality Expert Trajectories](https://mlsanity.com/paper/2310.10541/) (2023)
- [Scaling-Aware Data Selection for End-to-End Autonomous Driving Systems](https://mlsanity.com/paper/2604.08366/) (2026)
- [Q-TART: Quickly Training for Adversarial Robustness and in-Transferability](https://mlsanity.com/paper/2204.07024/) (2022)
- [Data Valuation for Offline Reinforcement Learning](https://mlsanity.com/paper/2205.09550/) (2022)
- [UDD: Dataset Distillation via Mining Underutilized Regions](https://mlsanity.com/paper/2408.16268/) (2024)
- [On the Importance of Pretraining Data Alignment for Atomic Property Prediction](https://mlsanity.com/paper/2502.11085/) (2025)
- [Iterative Misclassification Error Training (IMET): An Optimized Neural Network Training Technique for Image Classification](https://mlsanity.com/paper/2507.02979/) (2025)

## Abstract (arXiv)

When learning a new concept, not all training examples may prove equally useful for training: some may have higher or lower training value than others. The goal of this paper is to bring to the attention of the vision community the following considerations: (1) some examples are better than others for training detectors or classifiers, and (2) in the presence of better examples, some examples may negatively impact performance and removing them may be beneficial. In this paper, we propose an approach for measuring the training value of an example, and use it for ranking and greedily sorting examples. We test our methods on different vision tasks, models, datasets and classifiers. Our experiments show that the performance of current state-of-the-art detectors and classifiers can be improved when training on a subset, rather than the whole training set.