mlsanityData to 14 Oct 2026

On Data-centric Myths

arXiv 2111.11514 ↗First posted 22 Nov 2021cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

The community lacks theory-informed guidelines for building good data sets. We analyse theoretical directions relating to what aspects of the data matter and conclude that the intuitions derived from the existing literature are incorrect and misleading. Using empirical counter-examples, we show that 1) data dimension should not necessarily be minimised and 2) when manipulating data, preserving the distribution is inessential. This calls for a more data-aware theoretical understanding. Although not explored in this work, we propose the study of the impact of data modification on learned representations as a promising research direction.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 15 of its 19 references that are in the map. First posted on arXiv 22 Nov 2021.

Builds on

15 of 19 references.

References · 15

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  2. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  3. 2016Identity Mappings in Deep Residual Networkscited by 1,965
  4. 2019Domain Generalization by Solving Jigsaw Puzzlescited by 450
  5. 2020The Pitfalls of Simplicity Bias in Neural Networkscited by 351
  6. 2019Intrinsic dimension of data representations in deep neural networkscited by 282
  7. 2018Estimating the intrinsic dimension of datasets by a minimal neighborhood informationcited by 250
  8. 2018Deep learning generalizes because the parameter-function map is biased towards simple functionscited by 183
  9. 2020What shapes feature representations? Exploring datasets, architectures, and trainingcited by 126
  10. 2018Data Augmentation by Pairing Samples for Images Classificationcited by 96
  11. 2020On Mixup Regularizationcited by 83
  12. 2020Affinity and Diversity: Quantifying Mechanisms of Data Augmentationcited by 45
3 more
  1. 2020FMix: Enhancing Mixed Sample Data Augmentationcited by 11
  2. 2021Distributional Results for Model-Based Intrinsic Dimension Estimatorscited by 5
  3. 2018Bayesian Distance Clusteringcited by 4

Built on by

Cited by year

Citing papers per year: 2022: 4.

4 papers.

  1. 2022Is one annotation enough? A data-centric image classification benchmark for noisy and ambiguous label estimationcited by 43
  2. 2022Beyond Hard Labels: Investigating data label distributionscited by 7
  3. 2022Stochastic Perturbations of Tabular Features for Non-Deterministic Inference with Automungecited by 1
  4. 2022Geometric Regularization from Overparameterizationcited by 0