mlsanityData to 13 Oct 2026

Revisiting Self-Supervised Visual Representation Learning

arXiv 1901.09005 ↗First posted 25 Jan 2019cs.CVcited by 251

Not on the map. Why?

Abstract (arXiv)

Unsupervised visual representation learning remains a largely unsolved problem in computer vision research. Among a big body of recently proposed approaches for unsupervised learning of visual representations, a class of self-supervised techniques achieves superior performance on many challenging benchmarks. A large number of the pretext tasks for self-supervised learning have been studied, but other important aspects, such as the choice of convolutional neural networks (CNN), has not received equal attention. Therefore, we revisit numerous previously proposed self-supervised models, conduct a thorough large scale study and, as a result, uncover multiple crucial insights. We challenge a number of common practices in selfsupervised visual representation learning and observe that standard recipes for CNN design do not always translate to self-supervised representation learning. As part of our study, we drastically boost the performance of previously proposed techniques and outperform previously published state-of-the-art results by a large margin.

Key facts. Cited by 251 papers in the map (data through 13 Oct 2026). First posted on arXiv 25 Jan 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 97; 2022: 74; 2023: 38; 2024: 22; 2025: 15; 2026: 5.

251 papers.

Sort:

By area:

  1. 2022SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagerycited by 406
  2. 2021A Survey on Deep Semi-supervised Learningcited by 250
  3. 2021Self-Supervised Pretraining of 3D Features on any Point-Cloudcited by 217
  4. 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 205
  5. 2021Self-supervised Pretraining of Visual Features in the Wildcited by 197
  6. 2021LogME: Practical Assessment of Pre-trained Models for Transfer Learningcited by 168
  7. 2021Benchmarking Representation Learning for Natural World Image Collectionscited by 167
  8. 2021VideoMoCo: Contrastive Video Representation Learning with Temporally Adversarial Examplescited by 154
  9. 2022Backdoor Defense via Decoupling the Training Processcited by 152
  10. 2021Efficient Visual Pretraining with Contrastive Detectioncited by 150
  11. 2021Self-Supervised Representation Learning: Introduction, Advances and Challengescited by 147
  12. 2021Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Predictioncited by 137
38 more
  1. 2021SEED: Self-supervised Distillation For Visual Representationcited by 132
  2. 2022What to Hide from Your Students: Attention-Guided Masked Image Modelingcited by 117
  3. 2021AutoNovel: Automatically Discovering and Learning Novel Visual Categoriescited by 116
  4. 2021SiT: Self-supervised vIsion Transformercited by 99
  5. 2022Deep Unsupervised Domain Adaptation: A Review of Recent Advances and Perspectivescited by 98
  6. 2021Divide and Contrast: Self-supervised Learning from Uncurated Datacited by 85
  7. 2022Vision Models Are More Robust And Fair When Pretrained On Uncurated Images Without Supervisioncited by 80
  8. 2022When Do Flat Minima Optimizers Work?cited by 78
  9. 2022Divergence-aware Federated Self-Supervised Learningcited by 73
  10. 2023Battle of the Backbones: A Large-Scale Comparison of Pretrained Models across Computer Vision Taskscited by 72
  11. 2021Contrast to Divide: Self-Supervised Pre-Training for Learning with Noisy Labelscited by 69
  12. 2021Collaborative Unsupervised Visual Representation Learning from Decentralized Datacited by 69
  13. 2021Clusterability as an Alternative to Anchor Points When Learning with Noisy Labelscited by 65
  14. 2022Audio Self-supervised Learning: A Surveycited by 65
  15. 2021What Does Rotation Prediction Tell Us about Classifier Accuracy under Varying Testing Environments?cited by 63
  16. 2022Assessing the State of Self-Supervised Human Activity Recognition using Wearablescited by 63
  17. 2021Self-Supervised Learning of Remote Sensing Scene Representations Using Contrastive Multiview Codingcited by 58
  18. 2022Let Invariant Rationale Discovery Inspire Graph Contrastive Learningcited by 57
  19. 2021Pareto Self-Supervised Training for Few-Shot Learningcited by 52
  20. 2023Towards a Visual-Language Foundation Model for Computational Pathologycited by 52
  21. 2021Automated Self-Supervised Learning for Graphscited by 50
  22. 2021Understanding Negative Samples in Instance Discriminative Self-supervised Representation Learningcited by 47
  23. 2021Constrained Contrastive Distribution Learning for Unsupervised Anomaly Detection and Localisation in Medical Imagescited by 45
  24. 2021Training GANs with Stronger Augmentations via Contrastive Discriminatorcited by 45
  25. 2022Is one annotation enough? A data-centric image classification benchmark for noisy and ambiguous label estimationcited by 43
  26. 2021MiCE: Mixture of Contrastive Experts for Unsupervised Image Clusteringcited by 42
  27. 2023A General-Purpose Self-Supervised Model for Computational Pathologycited by 42
  28. 2021LiRA: Learning Visual Speech Representations from Audio through Self-supervisioncited by 41
  29. 2021Compressive Visual Representationscited by 41
  30. 2021Trash to Treasure: Harvesting OOD Data with Cross-Modal Matching for Open-Set Semi-Supervised Learningcited by 40
  31. 2022Spatial-Temporal Hypergraph Self-Supervised Learning for Crime Predictioncited by 39
  32. 2021Vectorization and Rasterization: Self-Supervised Learning for Sketch and Handwritingcited by 38
  33. 2021You Never Cluster Alonecited by 38
  34. 2022Benchmarking Omni-Vision Representation through the Lens of Visual Realmscited by 38
  35. 2022Self-Supervised Learning with an Information Maximization Criterioncited by 38
  36. 2021Geographical Knowledge-driven Representation Learning for Remote Sensing Imagescited by 37
  37. 2024SwitchTab: Switched Autoencoders Are Effective Tabular Learnerscited by 37
  38. 2021Revisiting the Transferability of Supervised Pretraining: an MLP Perspectivecited by 36

50 most cited of 251 All 251 (JSON)