mlsanityData to 7 Oct 2026

Billion-scale semi-supervised learning for image classification

arXiv 1905.00546 ↗First posted 2 May 2019cs.CVcited by 232

Not on the map. Why?

Abstract (arXiv)

This paper presents a study of semi-supervised learning with large convolutional networks. We propose a pipeline, based on a teacher/student paradigm, that leverages a large collection of unlabelled images (up to 1 billion). Our main goal is to improve the performance for a given target architecture, like ResNet-50 or ResNext. We provide an extensive analysis of the success factors of our approach, which leads us to formulate some recommendations to produce high-accuracy models for image classification with semi-supervised learning. As a result, our approach brings important gains to standard architectures for image, video and fine-grained classification. For instance, by leveraging one billion unlabelled images, our learned vanilla ResNet-50 achieves 81.2% top-1 accuracy on the ImageNet benchmark.

Key facts. Cited by 232 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 72; 2022: 74; 2023: 40; 2024: 24; 2025: 17; 2026: 5.

232 papers.

Sort:

By area:

  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  2. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  3. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  4. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,366
  5. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
  6. 2021Robust fine-tuning of zero-shot modelscited by 785
  7. 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 612
  8. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  9. 2021ResNet strikes back: An improved training procedure in timmcited by 365
  10. 2022MultiMAE: Multi-modal Multi-task Masked Autoencoderscited by 287
  11. 2023Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystackcited by 278
  12. 2021WebFace260M: A Benchmark Unveiling the Power of Million-Scale Deep Face Recognitioncited by 221
38 more
  1. 2021Accelerating Sparse Deep Neural Networkscited by 213
  2. 2021Partial success in closing the gap between human and machine visioncited by 185
  3. 2021ST++: Make Self-training Work Better for Semi-supervised Semantic Segmentationcited by 184
  4. 2021Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Bettercited by 153
  5. 2021Revisiting Point Cloud Shape Classification with a Simple and Effective Baselinecited by 145
  6. 2021Offboard 3D Object Detection from Point Cloud Sequencescited by 144
  7. 2021Ensemble of Averages: Improving Model Selection and Boosting Performance in Domain Generalizationcited by 144
  8. 2022Unsupervised Prompt Learning for Vision-Language Modelscited by 125
  9. 2021Re-labeling ImageNet: from Single to Multi-Labels, from Global to Localized Labelscited by 117
  10. 2022End-to-end Generative Pretraining for Multimodal Video Captioningcited by 115
  11. 2021On the Adversarial Robustness of Vision Transformerscited by 104
  12. 2021Compressing Visual-linguistic Model via Knowledge Distillationcited by 85
  13. 2021Multi-Task Self-Training for Learning General Representationscited by 85
  14. 2021Self-Supervised Pretraining Improves Self-Supervised Pretrainingcited by 81
  15. 2021Contrast to Divide: Self-Supervised Pre-Training for Learning with Noisy Labelscited by 69
  16. 2021Compute and Energy Consumption Trends in Deep Learning Inferencecited by 61
  17. 2022Continual Learning with Foundation Models: An Empirical Study of Latent Replaycited by 61
  18. 2022Data Feedback Loops: Model-driven Amplification of Dataset Biasescited by 59
  19. 2022Harmonizing the object recognition strategies of deep neural networks with humanscited by 59
  20. 2021Adversarial Robustness Comparison of Vision Transformer and MLP-Mixer to CNNscited by 57
  21. 2022Transferability in Deep Learning: A Surveycited by 57
  22. 2024Synthetic continued pretrainingcited by 56
  23. 2021Scalable and Adaptive Graph Neural Networks with Self-Label-Enhanced trainingcited by 54
  24. 2022Semi-Supervised and Unsupervised Deep Visual Learning: A Surveycited by 52
  25. 2021DASO: Distribution-Aware Semantics-Oriented Pseudo-label for Imbalanced Semi-Supervised Learningcited by 51
  26. 2022The Lie Derivative for Measuring Learned Equivariancecited by 51
  27. 2021Dynamic Distillation Network for Cross-Domain Few-Shot Recognition with Unlabeled Datacited by 49
  28. 2023SPOT: Self-Training with Patch-Order Permutation for Object-Centric Learning with Autoregressive Transformerscited by 49
  29. 2024Automatic Data Curation for Self-Supervised Learning: A Clustering-Based Approachcited by 48
  30. 2021SEAL: Self-supervised Embodied Active Learning using Exploration and 3D Consistencycited by 47
  31. 2021Data Poisoning Won't Save You From Facial Recognitioncited by 45
  32. 2022UCC: Uncertainty guided Cross-head Co-training for Semi-Supervised Semantic Segmentationcited by 45
  33. 2021SODA10M: A Large-Scale 2D Self/Semi-Supervised Object Detection Dataset for Autonomous Drivingcited by 43
  34. 2021Multilingual Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer of Vision-Language Modelscited by 41
  35. 2022WebFace260M: A Benchmark for Million-Scale Deep Face Recognitioncited by 41
  36. 2022Spurious Features Everywhere -- Large-Scale Detection of Harmful Spurious Features in ImageNetcited by 41
  37. 2022The Met Dataset: Instance-level Recognition for Artworkscited by 40
  38. 2021Supervised Transfer Learning at Scale for Medical Imagingcited by 38

50 most cited of 232 All 232 (JSON)