mlsanityData to 7 Oct 2026

Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey

arXiv 1902.06162 ↗First posted 16 Feb 2019cs.CVcited by 656

Not on the map. Why?

Abstract (arXiv)

Large-scale labeled data are generally required to train deep neural networks in order to obtain better performance in visual feature learning from images or videos for computer vision applications. To avoid extensive cost of collecting and annotating large-scale datasets, as a subset of unsupervised learning methods, self-supervised learning methods are proposed to learn general image and video features from large-scale unlabeled data without using any human-annotated labels. This paper provides an extensive review of deep learning-based self-supervised general visual feature learning methods from images or videos. First, the motivation, general pipeline, and terminologies of this field are described. Then the common deep neural network architectures that used for self-supervised learning are summarized. Next, the main components and evaluation metrics of self-supervised learning methods are reviewed followed by the commonly used image and video datasets and the existing self-supervised visual feature learning methods. Finally, quantitative performance comparisons of the reviewed methods on benchmark datasets are summarized and discussed for both image and video feature learning. At last, this paper is concluded and lists a set of promising future directions for self-supervised visual feature learning.

Key facts. Cited by 656 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Feb 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 171; 2022: 159; 2023: 130; 2024: 101; 2025: 57; 2026: 38.

656 papers.

Sort:

By area:

  1. 2021Transformers in Vision: A Surveycited by 1,009
  2. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 762
  3. 2021Model-Contrastive Federated Learningcited by 762
  4. 2021Domain Generalization: A Surveycited by 760
  5. 2021Deep Neural Networks and Tabular Data: A Surveycited by 489
  6. 2022SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagerycited by 402
  7. 2023A Comprehensive Survey on Test-Time Adaptation under Distribution Shiftscited by 353
  8. 2021Graph Self-Supervised Learning: A Surveycited by 289
  9. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 258
  10. 2021Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Modelscited by 251
  11. 2021A Survey on Deep Semi-supervised Learningcited by 250
  12. 2021A Geometric Analysis of Neural Collapse with Unconstrained Featurescited by 226
38 more
  1. 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 205
  2. 2021Unsupervised Semantic Segmentation by Contrasting Object Mask Proposalscited by 190
  3. 2021LogME: Practical Assessment of Pre-trained Models for Transfer Learningcited by 168
  4. 2022CroCo v2: Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flowcited by 165
  5. 2022CroCo: Self-Supervised Pre-training for 3D Vision Tasks by Cross-View Completioncited by 158
  6. 2023Prompt, Generate, then Cache: Cascade of Foundation Models makes Strong Few-shot Learnerscited by 152
  7. 2021Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyondcited by 151
  8. 2021SCARF: Self-Supervised Contrastive Learning using Random Feature Corruptioncited by 151
  9. 2022Backdoor Defense via Decoupling the Training Processcited by 149
  10. 2021Self-Supervised Representation Learning: Introduction, Advances and Challengescited by 147
  11. 2021Towards Faster and Stabilized GAN Training for High-fidelity Few-shot Image Synthesiscited by 140
  12. 2021Graph Learning: A Surveycited by 140
  13. 2022Self-supervised Learning in Remote Sensing: A Reviewcited by 137
  14. 2021Fine-Grained Image Analysis with Deep Learning: A Surveycited by 135
  15. 2021A Survey on Deep Learning Technique for Video Segmentationcited by 132
  16. 2022Pushing the Limits of Simple Pipelines for Few-Shot Learning: External Data and Fine-Tuning Make a Differencecited by 123
  17. 2021Toward Understanding the Feature Learning Process of Self-supervised Contrastive Learningcited by 119
  18. 2021Recent advances and clinical applications of deep learning in medical image analysiscited by 104
  19. 2021Improving Multi-hop Knowledge Base Question Answering by Learning Intermediate Supervision Signalscited by 100
  20. 2021Contrastive Learning with Stronger Augmentationscited by 100
  21. 2023Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospectscited by 97
  22. 2021Self-Supervised Transformer for Sparse and Irregularly Sampled Multivariate Clinical Time-Seriescited by 96
  23. 2022Neural Collapse: A Review on Modelling Principles and Generalizationcited by 95
  24. 2021Multi-Task Self-Training for Learning General Representationscited by 85
  25. 2022Self-Supervised Learning for Videos: A Surveycited by 85
  26. 2024Guiding Masked Representation Learning to Capture Spatio-Temporal Relationship of Electrocardiogramcited by 84
  27. 2021Prototypical Cross-domain Self-supervised Learning for Few-shot Unsupervised Domain Adaptationcited by 83
  28. 2021Exponential Moving Average Normalization for Self-supervised and Semi-supervised Learningcited by 78
  29. 2022Siamese Image Modeling for Self-Supervised Vision Representation Learningcited by 78
  30. 2021Self-supervised learning methods and applications in medical imaging analysis: A surveycited by 76
  31. 2025YOLO26: Key Architectural Enhancements and Performance Benchmarking for Real-Time Object Detectioncited by 76
  32. 2022A Comprehensive Survey on Deep Clustering: Taxonomy, Challenges, and Future Directionscited by 73
  33. 20233DSAM-adapter: Holistic adaptation of SAM from 2D to 3D for promptable tumor segmentationcited by 70
  34. 2021PointCutMix: Regularization Strategy for Point Cloud Classificationcited by 69
  35. 2021Kaleido-BERT: Vision-Language Pre-training on Fashion Domaincited by 67
  36. 2021Generative and Contrastive Self-Supervised Learning for Graph Anomaly Detectioncited by 67
  37. 2022Audio Self-supervised Learning: A Surveycited by 65
  38. 2022Video Transformers: A Surveycited by 63

50 most cited of 656 All 656 (JSON)