mlsanityData to 7 Oct 2026

Towards an Analytical Definition of Sufficient Data

arXiv 2202.03238 ↗First posted 7 Feb 2022cs.LG cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

We show that, for each of five datasets of increasing complexity, certain training samples are more informative of class membership than others. These samples can be identified a priori to training by analyzing their position in reduced dimensional space relative to the classes' centroids. Specifically, we demonstrate that samples nearer the classes' centroids are less informative than those that are furthest from it. For all five datasets, we show that there is no statistically significant difference between training on the entire training set and when excluding up to 2% of the data nearest to each class's centroid.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 27 of its 46 references that are in the map. First posted on arXiv 7 Feb 2022.

Builds on

27 of 46 references.

References · 27

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  5. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  6. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  7. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  8. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  9. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
  10. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,127
  11. 2021Scaling Vision Transformerscited by 993
  12. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 987
15 more
  1. 2021CvT: Introducing Convolutions to Vision Transformerscited by 946
  2. 2019Big Transfer (BiT): General Visual Representation Learningcited by 766
  3. 2021Scaling Vision with Sparse Mixture of Expertscited by 740
  4. 2019Large-Scale Long-Tailed Recognition in an Open Worldcited by 704
  5. 2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 566
  6. 2021CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windowscited by 526
  7. 2017A systematic study of the class imbalance problem in convolutional neural networkscited by 510
  8. 2020Meta Pseudo Labelscited by 372
  9. 2021High-Performance Large-Scale Image Recognition Without Normalizationcited by 359
  10. 2021TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?cited by 127
  11. 2017The Devil is in the Tails: Fine-grained Classification in the Wildcited by 89
  12. 2020Fixing the train-test resolution discrepancy: FixEfficientNetcited by 31
  13. 2020No Routing Needed Between Capsulescited by 27
  14. 2019Homogeneous Vector Capsules Enable Adaptive Gradient Descent in Convolutional Neural Networkscited by 4
  15. 2021On the Importance of Capturing a Sufficient Diversity of Perspective for the Classification of micro-PCBscited by 2

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 1.

3 papers.

  1. 2022Class Density and Dataset Quality in High-Dimensional, Unstructured Datacited by 1
  2. 2022Maintaining Performance with Less Datacited by 0
  3. 2023A Dataset Fusion Algorithm for Generalised Anomaly Detection in Homogeneous Periodic Time Series Datasetscited by 0