Exploring Dataset-Scale Indicators of Data Quality
Not on the map. Why?
Abstract (arXiv)
Modern computer vision foundation models are trained on massive amounts of data, incurring large economic and environmental costs. Recent research has suggested that improving data quality can significantly reduce the need for data quantity. But what constitutes data quality in computer vision? We posit that the quality of a given dataset can be decomposed into distinct sample-level and dataset-level constituents, and that the former have been more extensively studied than the latter. We ablate the effects of two important dataset-level constituents: label set design, and class balance. By monitoring these constituents using key indicators we provide, researchers and practitioners can better anticipate model performance, measured in terms of its accuracy and robustness to distribution shifts.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 24 of its 35 references that are in the map. First posted on arXiv 7 Nov 2023.
Builds on
24 of 35 references.
References · 24
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
- 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
- 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
- 2019Natural Adversarial Examplescited by 1,401
- 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
- 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
- 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 916
- 2021Florence: A New Foundation Model for Computer Visioncited by 802
- 2023DataComp: In search of the next generation of multimodal datasetscited by 635
12 more
- 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 604
- 2018Do Better ImageNet Models Transfer Better?cited by 560
- 2018The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scalecited by 542
- 2017A systematic study of the class imbalance problem in convolutional neural networkscited by 510
- 2022Data Distributional Properties Drive Emergent In-Context Learning in Transformerscited by 303
- 2023Semantic-SAM: Segment and Recognize Anything at Any Granularitycited by 185
- 2021Pre-training without Natural Imagescited by 100
- 2020The Cost of Training NLP Models: A Concise Overviewcited by 97
- 2022Quality Not Quantity: On the Interaction between Dataset Design and Robustness of CLIPcited by 97
- 2022Is a Caption Worth a Thousand Images? A Controlled Study for Representation Learningcited by 39
- 2020Representation Based Complexity Measures for Predicting Generalization in Deep Learningcited by 29
- 2023On the Theories Behind Hard Negative Sampling for Recommendationcited by 18
Built on by
Citing papers per year: 2024: 1.
1 paper.