mlsanityData to 14 Oct 2026

Understand and Improve Contrastive Learning Methods for Visual Representation: A Review

arXiv 2106.03259 ↗First posted 6 Jun 2021cs.LG cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

Traditional supervised learning methods are hitting a bottleneck because of their dependency on expensive manually labeled data and their weaknesses such as limited generalization ability and vulnerability to adversarial attacks. A promising alternative, self-supervised learning, as a type of unsupervised learning, has gained popularity because of its potential to learn effective data representations without manual labeling. Among self-supervised learning algorithms, contrastive learning has achieved state-of-the-art performance in several fields of research. This literature review aims to provide an up-to-date analysis of the efforts of researchers to understand the key components and the limitations of self-supervised learning.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 46 of its 57 references that are in the map. First posted on arXiv 6 Jun 2021.

Builds on

46 of 57 references.

References · 46

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  5. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  6. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  7. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
  8. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  9. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  10. 2020Supervised Contrastive Learningcited by 2,829
  11. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  12. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
34 more
  1. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
  2. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  3. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  4. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
  5. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,380
  6. 2019Contrastive Multiview Codingcited by 1,301
  7. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,273
  8. 2016Colorful Image Colorizationcited by 1,114
  9. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,066
  10. 2020What Makes for Good Views for Contrastive Learning?cited by 846
  11. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 808
  12. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 734
  13. 2020Self-supervised Learning: Generative or Contrastivecited by 694
  14. 2019Self-supervised Visual Feature Learning with Deep Neural Networks: A Surveycited by 657
  15. 2020A Survey on Contrastive Self-supervised Learningcited by 554
  16. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 459
  17. 2018Invariant Information Clustering for Unsupervised Image Classification and Segmentationcited by 455
  18. 2015Unsupervised Learning of Video Representations using LSTMscited by 435
  19. 2020Hard Negative Mixing for Contrastive Learningcited by 402
  20. 2020Spatiotemporal Contrastive Video Representation Learningcited by 372
  21. 2019S4L: Self-Supervised Semi-Supervised Learningcited by 335
  22. 2014Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networkscited by 300
  23. 2016Learning Representations for Automatic Colorizationcited by 251
  24. 2020What Should Not Be Contrastive in Contrastive Learningcited by 226
  25. 2020How Neural Networks Extrapolate: From Feedforward to Graph Neural Networkscited by 222
  26. 2017Unsupervised Representation Learning by Sorting Sequencescited by 221
  27. 2018Self-Supervised Video Representation Learning with Space-Time Cubic Puzzlescited by 149
  28. 2020Intriguing Properties of Contrastive Lossescited by 130
  29. 2020Contrastive Learning with Adversarial Examplescited by 91
  30. 2019A Mutual Information Maximization Perspective of Language Representation Learningcited by 85
  31. 2019AET vs. AED: Unsupervised Representation Learning by Auto-Encoding Transformations rather than Datacited by 77
  32. 2020Subject-Aware Contrastive Learning for Biosignalscited by 76
  33. 2019Self-Supervised Learning for Cardiac MR Image Segmentation by Anatomical Position Predictioncited by 70
  34. 2021Mine Your Own vieW: Self-Supervised Learning Through Across-Sample Predictioncited by 36

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 4; 2024: 0; 2025: 1.

7 papers.

  1. 2022EquiMod: An Equivariance Module to Improve Self-Supervised Learningcited by 21
  2. 2023Contrastive Multi-view Subspace Clustering of Hyperspectral Images based on Graph Convolutional Networkscited by 11
  3. 2022Hyperspectral Image Classification With Contrastive Graph Convolutional Networkcited by 8
  4. 2023Guarding Barlow Twins Against Overfitting with Mixed Samplescited by 8
  5. 2023Interpretable Alzheimer's Disease Classification Via a Contrastive Diffusion Autoencodercited by 3
  6. 2023Damage GAN: A Generative Model for Imbalanced Datacited by 0
  7. 2025Improve Contrastive Clustering Performance by Multiple Fusing-Augmenting ViT Blockscited by 0