mlsanityData to 7 Oct 2026

A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends

arXiv 2301.05712 ↗First posted 13 Jan 2023cs.LGcited by 205

Not on the map. Why?

Abstract (arXiv)

Deep supervised learning algorithms typically require a large volume of labeled data to achieve satisfactory performance. However, the process of collecting and labeling such data can be expensive and time-consuming. Self-supervised learning (SSL), a subset of unsupervised learning, aims to learn discriminative features from unlabeled data without relying on human-annotated labels. SSL has garnered significant attention recently, leading to the development of numerous related algorithms. However, there is a dearth of comprehensive studies that elucidate the connections and evolution of different SSL variants. This paper presents a review of diverse SSL methods, encompassing algorithmic aspects, application domains, three key trends, and open research questions. Firstly, we provide a detailed introduction to the motivations behind most SSL algorithms and compare their commonalities and differences. Secondly, we explore representative applications of SSL in domains such as image processing, computer vision, and natural language processing. Lastly, we discuss the three primary trends observed in SSL research and highlight the open questions that remain. A curated collection of valuable resources can be accessed at https://github.com/guijiejie/SSL.

Key facts. Cited by 205 papers in the map (data through 7 Oct 2026). Builds on 254 of its 309 references that are in the map. First posted on arXiv 13 Jan 2023.

Builds on

254 of 309 references.

References · 254

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  7. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  8. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  9. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  10. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  11. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  12. 2021Zero-Shot Text-to-Image Generationcited by 4,276
38 more
  1. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  2. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  3. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  4. 2020Supervised Contrastive Learningcited by 2,803
  5. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  6. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  7. 2017The Kinetics Human Action Video Datasetcited by 1,894
  8. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  9. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  10. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  11. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  12. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  13. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,393
  14. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  15. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  16. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
  17. 2019Contrastive Multiview Codingcited by 1,297
  18. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,215
  19. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
  20. 2016Colorful Image Colorizationcited by 1,113
  21. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,092
  22. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,063
  23. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  24. 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,020
  25. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,013
  26. 2019Strategies for Pre-training Graph Neural Networkscited by 964
  27. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
  28. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  29. 2020What Makes for Good Views for Contrastive Learning?cited by 843
  30. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 806
  31. 2015Unsupervised Deep Embedding for Clustering Analysiscited by 763
  32. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 723
  33. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 718
  34. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 703
  35. 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 702
  36. 2020Self-supervised Learning: Generative or Contrastivecited by 693
  37. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 685
  38. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 672

50 most cited of 254 All 254 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 44; 2025: 94; 2026: 64.

205 papers.

Sort:

By area:

  1. 2023Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospectscited by 97
  2. 2023Visual Point Cloud Forecasting enables Scalable Autonomous Drivingcited by 97
  3. 2024Large Language Models are Strong Audio-Visual Speech Recognition Learnerscited by 40
  4. 2025Spatial-SSRL: Enhancing Spatial Understanding via Self-Supervised Reinforcement Learningcited by 38
  5. 2025Jigsaw-R1: A Study of Rule-based Visual Reinforcement Learning with Jigsaw Puzzlescited by 36
  6. 2025Joint Embedding vs Reconstruction: Provable Benefits of Latent Space Prediction for Self Supervised Learningcited by 33
  7. 2024Deep Learning for Video Anomaly Detection: A Reviewcited by 27
  8. 2025Perch 2.0: The Bittern Lesson for Bioacousticscited by 25
  9. 2025The Evolution of Dataset Distillation: Toward Scalable and Generalizable Solutionscited by 24
  10. 2025Advances in Multimodal Adaptation and Generalization: From Traditional Approaches to Foundation Modelscited by 23
  11. 2024FRoundation: Are Foundation Models Ready for Face Recognition?cited by 22
  12. 2024An overview of domain-specific foundation model: key technologies, applications and challengescited by 19
38 more
  1. 2024Spatial Structure Constraints for Weakly Supervised Semantic Segmentationcited by 17
  2. 2024MiM: Mask in Mask Self-Supervised Pre-Training for 3D Medical Image Analysiscited by 17
  3. 2024COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-trainingcited by 15
  4. 2024Salvaging the Overlooked: Leveraging Class-Aware Contrastive Learning for Multi-Class Anomaly Detectioncited by 15
  5. 2025A Survey of Pathology Foundation Model: Progress and Future Directionscited by 14
  6. 2024Promoting cross-modal representations to improve multimodal foundation models for physiological signalscited by 13
  7. 2025Stealthy Backdoor Attack in Self-Supervised Learning Vision Encoders for Large Vision Language Modelscited by 13
  8. 2024A Survey on Point-of-Interest Recommendation: Models, Architectures, and Securitycited by 12
  9. 2025General Intelligence Requires Reward-based Pretrainingcited by 12
  10. 2026INSID3: Training-Free In-Context Segmentation with DINOv3cited by 12
  11. 2025Large Wireless Localization Model (LWLM): A Foundation Model for Positioning in 6G Networkscited by 10
  12. 2025Scaling and Enhancing LLM-based AVSR: A Sparse Mixture of Projectors Approachcited by 10
  13. 2025Pelican-VL 1.0: A Foundation Brain Model for Embodied Intelligencecited by 10
  14. 2024A New Era in Computational Pathology: A Survey on Foundation and Vision-Language Modelscited by 9
  15. 2024Forte : Finding Outliers with Representation Typicality Estimationcited by 9
  16. 2025Training-free Detection of AI-generated images via Cropping Robustnesscited by 9
  17. 2024Can Transformers Do Enumerative Geometry?cited by 8
  18. 2024A Survey of the Self Supervised Learning Mechanisms for Vision Transformerscited by 8
  19. 2025Scaling Self-Supervised Representation Learning for Symbolic Piano Performancecited by 7
  20. 2024Evolving from Single-modal to Multi-modal Facial Deepfake Detection: Progress and Challengescited by 6
  21. 2024DeDe: Detecting Backdoor Samples for SSL Encoders via Decoderscited by 6
  22. 2024Multi-objective Deep Learning: Taxonomy and Survey of the State of the Artcited by 6
  23. 2025Revisiting semi-supervised learning in the era of foundation modelscited by 6
  24. 2025Beyond Degradation Redundancy: Contrastive Prompt Learning for All-in-One Image Restorationcited by 6
  25. 2025RINO: Renormalization Group Invariance with No Labelscited by 6
  26. 2024Invisible Backdoor Attack against Self-supervised Learningcited by 5
  27. 2025Deep Learning in Palmprint Recognition-A Comprehensive Surveycited by 5
  28. 2025SSL4Eco: A Global Seasonal Dataset for Geospatial Foundation Models in Ecologycited by 5
  29. 2024Leveraging Self-Supervised Learning for Scene Classification in Child Sexual Abuse Imagerycited by 4
  30. 2024CroMo-Mixup: Augmenting Cross-Model Representations for Continual Self-Supervised Learningcited by 4
  31. 2025Masked Scene Modeling: Narrowing the Gap Between Supervised and Self-Supervised Learning in 3D Scene Understandingcited by 4
  32. 2025Agentic AI for autonomous anomaly management in complex systemscited by 4
  33. 2025CoUn: Empowering Machine Unlearning via Contrastive Learningcited by 4
  34. 2025Self-Supervised Multi-Scale Transformer with Attention-Guided Fusion for Efficient Crack Detectioncited by 4
  35. 2026Understanding Self-Supervised Learning via Latent Distribution Matchingcited by 4
  36. 2024A Literature Review of Literature Reviews in Pattern Analysis and Machine Intelligencecited by 3
  37. 2024Tabular Data Contrastive Learning via Class-Conditioned and Feature-Correlation Based Augmentationcited by 3
  38. 2024Score-based Generative Priors Guided Model-driven Network for MRI Reconstructioncited by 3

50 most cited of 205 All 205 (JSON)