mlsanityData to 14 Oct 2026

A Survey on Self-Supervised Representation Learning

arXiv 2308.11455 ↗First posted 22 Aug 2023cs.LG cs.AI eess.IVcited by 9

Not on the map. Why?

Abstract (arXiv)

Learning meaningful representations is at the heart of many tasks in the field of modern machine learning. Recently, a lot of methods were introduced that allow learning of image representations without supervision. These representations can then be used in downstream tasks like classification or object detection. The quality of these representations is close to supervised learning, while no labeled images are needed. This survey paper provides a comprehensive review of these methods in a unified notation, points out similarities and differences of these methods, and proposes a taxonomy which sets these methods in relation to each other. Furthermore, our survey summarizes the most-recent experimental results reported in the literature in form of a meta-study. Our survey is intended as a starting point for researchers and practitioners who want to dive into the field of representation learning.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 51 of its 73 references that are in the map. First posted on arXiv 22 Aug 2023.

Builds on

51 of 73 references.

References · 51

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  6. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  7. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  8. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  9. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  10. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  11. 2017Mask R-CNNcited by 7,277
  12. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
38 more
  1. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  2. 2016Layer Normalizationcited by 5,669
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  4. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  5. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  6. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  7. 2013Auto-Encoding Variational Bayescited by 4,053
  8. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  9. 2015Continuous control with deep reinforcement learningcited by 3,407
  10. 2015Fast R-CNNcited by 2,933
  11. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  12. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  13. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,316
  14. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  15. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
  16. 2016node2vec: Scalable Feature Learning for Networkscited by 1,918
  17. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
  18. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,838
  19. 2020Knowledge Distillation: A Surveycited by 1,603
  20. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  21. 2014DeepWalk: Online Learning of Social Representationscited by 1,547
  22. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
  23. 2016Variational Graph Auto-Encoderscited by 1,382
  24. 2019Contrastive Multiview Codingcited by 1,301
  25. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,057
  26. 2016Conditional Image Generation with PixelCNN Decoderscited by 862
  27. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 808
  28. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 673
  29. 2013Network In Networkcited by 605
  30. 2017Large Batch Training of Convolutional Networkscited by 535
  31. 2019Self-labelling via simultaneous clustering and representation learningcited by 528
  32. 2021Time-Series Representation Learning via Temporal and Contextual Contrastingcited by 345
  33. 2020Whitening for Self-Supervised Representation Learningcited by 279
  34. 2019Revisiting Self-Supervised Visual Representation Learningcited by 251
  35. 2021Scaling Local Self-Attention for Parameter Efficient Visual Backbonescited by 220
  36. 2019Scaling and Benchmarking Self-Supervised Visual Representation Learningcited by 201
  37. 2021Efficient Self-supervised Vision Transformers for Representation Learningcited by 149
  38. 2022Image Data Augmentation for Deep Learning: A Surveycited by 96

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 3.

9 papers.

  1. 2025AIM-Fair: Advancing Algorithmic Fairness via Selectively Fine-Tuning Biased Models with Contextual Synthetic Datacited by 5
  2. 2026Zero-Flow Encoderscited by 4
  3. 2024SInViG: A Self-Evolving Interactive Visual Agent for Human-Robot Interactioncited by 2
  4. 2025Joint Embedding Predictive Architecture for self-supervised pretraining on polymer molecular graphscited by 1
  5. 2025Oh-A-DINO: Understanding and Enhancing Attribute-Level Information in Self-Supervised Object-Centric Representationscited by 0
  6. 2025SARL: Spatially-Aware Self-Supervised Representation Learning for Visuo-Tactile Perceptioncited by 0
  7. 2025Osmotic Learning: A Self-Supervised Paradigm for Decentralized Contextual Data Representationcited by 0
  8. 2026The Loss Is Not Enough: Sampling Conditions and Inductive Bias in Contrastive Representation Learningcited by 0
  9. 2026Graph Neural Networks for Semi-Supervised Image Classification with Multi-Feature Aggregationcited by 0