# Flaws of ImageNet, Computer Vision's Favourite Dataset

arXiv: https://arxiv.org/abs/2412.00076 · HTML: https://mlsanity.com/paper/2412.00076/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 6 references that are in the map.
- First posted on arXiv 26 Nov 2024.

## Builds on (most cited first, 6 of 6)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Confident Learning: Estimating Uncertainty in Dataset Labels](https://mlsanity.com/paper/1911.00068/) (2019)
- [Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks](https://mlsanity.com/paper/2103.14749/) (2021)
- [Bugs in the Data: How ImageNet Misrepresents Biodiversity](https://mlsanity.com/paper/2208.11695/) (2022)

## Built on by (most cited first, 12 of 12)

- [DRAGON: A Large-Scale Dataset of Realistic Images Generated by Diffusion Models](https://mlsanity.com/paper/2505.11257/) (2025)
- [Multiplicity is an Inevitable and Inherent Challenge in Multimodal Learning](https://mlsanity.com/paper/2505.19614/) (2025)
- [I Am Big, You Are Little; I Am Right, You Are Wrong](https://mlsanity.com/paper/2507.23509/) (2025)
- [Image Recognition with Vision and Language Embeddings of VLMs](https://mlsanity.com/paper/2509.09311/) (2025)
- [Multimodal Large Language Models as Image Classifiers](https://mlsanity.com/paper/2603.06578/) (2026)
- [Robust Context-Aware Object Recognition](https://mlsanity.com/paper/2510.00618/) (2025)
- [Learning on the Fly: Replay-Based Continual Object Perception for Indoor Drones](https://mlsanity.com/paper/2602.13440/) (2026)
- [Doomed to Re-Annotate, Forever: The ImageNet Story](https://mlsanity.com/paper/2608.13783/) (2026)
- [Meta-Learning Transformers to Improve In-Context Generalization](https://mlsanity.com/paper/2507.05019/) (2025)
- [Koo-Fu CLIP: Closed-Form Adaptation of Vision-Language Models via Fukunaga-Koontz Linear Discriminant Analysis](https://mlsanity.com/paper/2602.01127/) (2026)
- [Beyond Loss Values: Robust Dynamic Pruning via Loss Trajectory Alignment](https://mlsanity.com/paper/2604.07306/) (2026)
- [Why Domain Matters: Domain-Aware Benchmarking of Underwater Object Detection and Annotation Quality](https://mlsanity.com/paper/2607.10575/) (2026)

## Abstract (arXiv)

Since its release, ImageNet-1k dataset has become a gold standard for evaluating model performance. It has served as the foundation for numerous other datasets and training tasks in computer vision. As models have improved in accuracy, issues related to label correctness have become increasingly apparent. In this blog post, we analyze the issues in the ImageNet-1k dataset, including incorrect labels, overlapping or ambiguous class definitions, training-evaluation domain shifts, and image duplicates. The solutions for some problems are straightforward. For others, we hope to start a broader conversation about refining this influential dataset to better serve future research.