mlsanityData to 7 Oct 2026

Flaws of ImageNet, Computer Vision's Favourite Dataset

arXiv 2412.00076 ↗First posted 26 Nov 2024cs.CVcited by 12

Not on the map. Why?

Abstract (arXiv)

Since its release, ImageNet-1k dataset has become a gold standard for evaluating model performance. It has served as the foundation for numerous other datasets and training tasks in computer vision. As models have improved in accuracy, issues related to label correctness have become increasingly apparent. In this blog post, we analyze the issues in the ImageNet-1k dataset, including incorrect labels, overlapping or ambiguous class definitions, training-evaluation domain shifts, and image duplicates. The solutions for some problems are straightforward. For others, we hope to start a broader conversation about refining this influential dataset to better serve future research.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 6 of its 6 references that are in the map. First posted on arXiv 26 Nov 2024.

Builds on

6 of 6 references.

References · 6

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  3. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
  4. 2019Confident Learning: Estimating Uncertainty in Dataset Labelscited by 411
  5. 2021Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarkscited by 388
  6. 2022Bugs in the Data: How ImageNet Misrepresents Biodiversitycited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 6.

12 papers.

  1. 2025DRAGON: A Large-Scale Dataset of Realistic Images Generated by Diffusion Modelscited by 4
  2. 2025Multiplicity is an Inevitable and Inherent Challenge in Multimodal Learningcited by 4
  3. 2025I Am Big, You Are Little; I Am Right, You Are Wrongcited by 3
  4. 2025Image Recognition with Vision and Language Embeddings of VLMscited by 2
  5. 2026Multimodal Large Language Models as Image Classifierscited by 2
  6. 2025Robust Context-Aware Object Recognitioncited by 1
  7. 2026Learning on the Fly: Replay-Based Continual Object Perception for Indoor Dronescited by 1
  8. 2026Doomed to Re-Annotate, Forever: The ImageNet Storycited by 1
  9. 2025Meta-Learning Transformers to Improve In-Context Generalizationcited by 0
  10. 2026Koo-Fu CLIP: Closed-Form Adaptation of Vision-Language Models via Fukunaga-Koontz Linear Discriminant Analysiscited by 0
  11. 2026Beyond Loss Values: Robust Dynamic Pruning via Loss Trajectory Alignmentcited by 0
  12. 2026Why Domain Matters: Domain-Aware Benchmarking of Underwater Object Detection and Annotation Qualitycited by 0