mlsanityData to 14 Oct 2026

Know Your Self-supervised Learning: A Survey on Image-based Generative and Discriminative Training

arXiv 2305.13689 ↗First posted 23 May 2023cs.CV cs.AIcited by 39

Not on the map. Why?

Abstract (arXiv)

Although supervised learning has been highly successful in improving the state-of-the-art in the domain of image-based computer vision in the past, the margin of improvement has diminished significantly in recent years, indicating that a plateau is in sight. Meanwhile, the use of self-supervised learning (SSL) for the purpose of natural language processing (NLP) has seen tremendous successes during the past couple of years, with this new learning paradigm yielding powerful language models. Inspired by the excellent results obtained in the field of NLP, self-supervised methods that rely on clustering, contrastive learning, distillation, and information-maximization, which all fall under the banner of discriminative SSL, have experienced a swift uptake in the area of computer vision. Shortly afterwards, generative SSL frameworks that are mostly based on masked image modeling, complemented and surpassed the results obtained with discriminative SSL. Consequently, within a span of three years, over 100 unique general-purpose frameworks for generative and discriminative SSL, with a focus on imaging, were proposed. In this survey, we review a plethora of research efforts conducted on image-oriented SSL, providing a historic view and paying attention to best practices as well as useful software packages. While doing so, we discuss pretext tasks for image-based SSL, as well as techniques that are commonly used in image-based SSL. Lastly, to aid researchers who aim at contributing to image-focused SSL, we outline a number of promising research directions.

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). Builds on 172 of its 229 references that are in the map. First posted on arXiv 23 May 2023.

Builds on

172 of 229 references.

References · 172

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  5. 2020Language Models are Few-Shot Learnerscited by 32,419
  6. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  7. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  8. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  9. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  10. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  11. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  12. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
38 more
  1. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  2. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  5. 2017Inductive Representation Learning on Large Graphscited by 6,320
  6. 2022Generative Adversarial Networkscited by 5,627
  7. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  8. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  9. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  10. 2014Going Deeper with Convolutionscited by 4,241
  11. 2022A ConvNet for the 2020scited by 3,579
  12. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  13. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
  14. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  15. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  16. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  17. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  18. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,316
  19. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  20. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
  21. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,838
  22. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
  23. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  24. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,428
  25. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
  26. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,380
  27. 2019Contrastive Multiview Codingcited by 1,301
  28. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,273
  29. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,153
  30. 2016Colorful Image Colorizationcited by 1,114
  31. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,070
  32. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,066
  33. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,057
  34. 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,028
  35. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 903
  36. 2020What Makes for Good Views for Contrastive Learning?cited by 846
  37. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 808
  38. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777

50 most cited of 172 All 172 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 9; 2024: 14; 2025: 15; 2026: 1.

39 papers.

Sort:

By area:

  1. 2023Multimodal Foundation Models: From Specialists to General-Purpose Assistantscited by 247
  2. 2024FSFM: A Generalizable Face Security Foundation Model via Self-Supervised Facial Representation Learningcited by 23
  3. 2023Large Sequence Models for Sequential Decision-Making: A Surveycited by 19
  4. 2023Knowledge Transfer from Vision Foundation Models for Efficient Training of Small Task-specific Modelscited by 15
  5. 2024MEGA: Masked Generative Autoencoder for Human Mesh Recoverycited by 14
  6. 2025Handwritten Text Recognition: A Surveycited by 12
  7. 2023Intrinsic Self-Supervision for Data Quality Auditscited by 8
  8. 2025Evaluating Visual Explanations of Attention Maps for Transformer-based Medical Imagingcited by 7
  9. 2023Towards Reliable Dermatology Evaluation Benchmarkscited by 6
  10. 2024Automated Contrastive Learning Strategy Search for Time Seriescited by 5
  11. 2024An Experimental Study on Exploring Strong Lightweight Vision Transformers via Masked Image Modeling Pre-Trainingcited by 4
  12. 2024Self-Supervised Learning for Text Recognition: A Critical Surveycited by 3
27 more
  1. 2025Clustering via Self-Supervised Diffusioncited by 3
  2. 2023Augmentation-aware Self-supervised Learning with Conditioned Projectorcited by 2
  3. 2024Leveraging Human-Machine Interactions for Computer Vision Dataset Quality Enhancementcited by 2
  4. 2024Self-Supervised Learning for Real-World Object Detection: a Surveycited by 2
  5. 2024On Discriminative Probabilistic Modeling for Self-Supervised Representation Learningcited by 2
  6. 2025Exploring Patient Data Requirements in Training Effective AI Models for MRI-based Breast Cancer Classificationcited by 2
  7. 2025How PARTs assemble into wholes: Learning the relative composition of imagescited by 2
  8. 2023MNN: Mixed Nearest-Neighbors for Self-Supervised Learningcited by 1
  9. 2023Simple but Effective Unsupervised Classification for Specified Domain Images: A Case Study on Fungi Imagescited by 1
  10. 2024Low-Rank Approximation of Structural Redundancy for Self-Supervised Learningcited by 1
  11. 2024Spatial Context-based Self-Supervised Learning for Handwritten Text Recognitioncited by 1
  12. 2025Self-supervised Benchmark Lottery on ImageNet: Do Marginal Improvements Translate to Improvements on Similar Datasets?cited by 1
  13. 2025Color Flow Imaging Microscopy Improves Identification of Stress Sources of Protein Aggregates in Biopharmaceuticalscited by 1
  14. 2025Adversarial Robustness of Discriminative Self-Supervised Learning in Visioncited by 1
  15. 2023Unveiling the Potential of Probabilistic Embeddings in Self-Supervised Learningcited by 0
  16. 2024The Bad Batches: Enhancing Self-Supervised Learning in Image Classification Through Representative Batch Curationcited by 0
  17. 2024Self-supervised learning for classifying paranasal anomalies in the maxillary sinuscited by 0
  18. 2024Revisiting Face Forgery Detection: From Facial Representation to Forgery Detectioncited by 0
  19. 2024BioNCERE: Non-Contrastive Enhancement For Relation Extraction In Biomedical Textscited by 0
  20. 2025Escaping The Big Data Paradigm in Self-Supervised Representation Learningcited by 0
  21. 2025One Patient's Annotation is Another One's Initialization: Towards Zero-Shot Surgical Video Segmentation with Cross-Patient Initializationcited by 0
  22. 2025Conjuring Positive Pairs for Efficient Unification of Representation Learning and Image Synthesiscited by 0
  23. 2025Leveraging Self-Supervised Features for Efficient Flooded Region Identification in UAV Aerial Imagescited by 0
  24. 2025Token-Based Detection of Spurious Correlations in Vision Transformerscited by 0
  25. 2025Unsupervised Transformer Pre-Training for Images: Self-Distillation, Mean Teachers, and Random Cropscited by 0
  26. 2025Understanding Self-supervised Contrastive Learning through Supervised Objectivescited by 0
  27. 2026Lifelong Representations: A Survey on Continual Self-Supervised Learning for Vision Modelscited by 0