mlsanityData to 7 Oct 2026

Are we done with ImageNet?

arXiv 2006.07159 ↗First posted 12 Jun 2020cs.CV cs.LGcited by 329

Not on the map. Why?

Abstract (arXiv)

Yes, and no. We ask whether recent progress on the ImageNet classification benchmark continues to represent meaningful generalization, or whether the community has started to overfit to the idiosyncrasies of its labeling procedure. We therefore develop a significantly more robust procedure for collecting human annotations of the ImageNet validation set. Using these new labels, we reassess the accuracy of recently proposed ImageNet classifiers, and find their gains to be substantially smaller than those reported on the original labels. Furthermore, we find the original ImageNet labels to no longer be the best predictors of this independently-collected set, indicating that their usefulness in evaluating vision models may be nearing an end. Nevertheless, we find our annotation procedure to have largely remedied the errors in the original labels, reinforcing ImageNet as a powerful benchmark for future research in visual recognition.

Key facts. Cited by 329 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 65; 2022: 67; 2023: 73; 2024: 56; 2025: 45; 2026: 23.

329 papers.

Sort:

By area:

  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  2. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  3. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  4. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  5. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  6. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,366
  7. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
  8. 2021Scaling Vision Transformerscited by 993
  9. 2021CvT: Introducing Convolutions to Vision Transformerscited by 946
  10. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 762
  11. 2021Going deeper with Image Transformerscited by 759
  12. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 690
38 more
  1. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 685
  2. 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 572
  3. 2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
  4. 2023Symbolic Discovery of Optimization Algorithmscited by 423
  5. 2021Conditional Positional Encodings for Vision Transformerscited by 402
  6. 2023EVA-02: A Visual Representation for Neon Genesiscited by 377
  7. 2021ResNet strikes back: An improved training procedure in timmcited by 365
  8. 2021ResMLP: Feedforward networks for image classification with data-efficient trainingcited by 338
  9. 2022What does a platypus look like? Generating customized prompts for zero-shot image classificationcited by 263
  10. 2021When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentationscited by 259
  11. 2021Global Filter Networks for Image Classificationcited by 257
  12. 2023PaLI-X: On Scaling up a Multilingual Vision and Language Modelcited by 254
  13. 2021Incorporating Convolution Designs into Visual Transformerscited by 248
  14. 2022TinyViT: Fast Pretraining Distillation for Small Vision Transformerscited by 232
  15. 2023EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attentioncited by 172
  16. 2022Surrogate Gap Minimization Improves Sharpness-Aware Trainingcited by 171
  17. 2021VOLO: Vision Outlooker for Visual Recognitioncited by 164
  18. 2022FlexiViT: One Model for All Patch Sizescited by 151
  19. 2021All Tokens Matter: Token Labeling for Training Better Vision Transformerscited by 144
  20. 2024PlainMamba: Improving Non-Hierarchical Mamba in Visual Recognitioncited by 141
  21. 2021The Benchmark Lotterycited by 133
  22. 2021Toward a Perspectivist Turn in Ground Truthing for Predictive Computingcited by 129
  23. 2021TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?cited by 127
  24. 2023PaLI-3 Vision Language Models: Smaller, Faster, Strongercited by 125
  25. 2021Re-labeling ImageNet: from Single to Multi-Labels, from Global to Localized Labelscited by 117
  26. 2022ViTAEv2: Vision Transformer Advanced by Exploring Inductive Bias for Image Recognition and Beyondcited by 117
  27. 2021Vision Permutator: A Permutable MLP-Like Architecture for Visual Recognitioncited by 103
  28. 2021Long-Short Transformer: Efficient Transformers for Language and Visioncited by 102
  29. 2022Plex: Towards Reliability using Pretrained Large Model Extensionscited by 102
  30. 2021Reduced, Reused and Recycled: The Life of a Dataset in Machine Learning Researchcited by 100
  31. 2021Large-scale Unsupervised Semantic Segmentationcited by 98
  32. 2022MiniViT: Compressing Vision Transformers with Weight Multiplexingcited by 88
  33. 2023Getting ViT in Shape: Scaling Laws for Compute-Optimal Model Designcited by 87
  34. 2024Model Stock: All we need is just a few fine-tuned modelscited by 87
  35. 2023A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinkingcited by 85
  36. 2022Distilling Model Failures as Directions in Latent Spacecited by 82
  37. 2022Vision Models Are More Robust And Fair When Pretrained On Uncurated Images Without Supervisioncited by 80
  38. 2024Benchmarking Uncertainty Disentanglement: Specialized Uncertainties for Specialized Taskscited by 78

50 most cited of 329 All 329 (JSON)