mlsanityData to 14 Oct 2026

Weakly Supervised Contrastive Learning

arXiv 2110.04770 ↗First posted 10 Oct 2021cs.CVcited by 81

Not on the map. Why?

Abstract (arXiv)

Unsupervised visual representation learning has gained much attention from the computer vision community because of the recent achievement of contrastive learning. Most of the existing contrastive learning frameworks adopt the instance discrimination as the pretext task, which treating every single instance as a different class. However, such method will inevitably cause class collision problems, which hurts the quality of the learned representation. Motivated by this observation, we introduced a weakly supervised contrastive learning framework (WCL) to tackle this issue. Specifically, our proposed framework is based on two projection heads, one of which will perform the regular instance discrimination task. The other head will use a graph-based method to explore similar samples and generate a weak label, then perform a supervised contrastive learning task based on the weak label to pull the similar images closer. We further introduced a K-Nearest Neighbor based multi-crop strategy to expand the number of positive samples. Extensive experimental results demonstrate WCL improves the quality of self-supervised representations across different datasets. Notably, we get a new state-of-the-art result for semi-supervised learning. With only 1% and 10% labeled examples, WCL achieves 65% and 72% ImageNet Top-1 Accuracy using ResNet50, which is even higher than SimCLRv2 with ResNet101.

Key facts. Cited by 81 papers in the map (data through 14 Oct 2026). Builds on 41 of its 57 references that are in the map. First posted on arXiv 10 Oct 2021.

Builds on

41 of 57 references.

References · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  6. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  7. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  8. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  9. 2020Supervised Contrastive Learningcited by 2,829
  10. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  11. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  12. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,085
29 more
  1. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,908
  2. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
  3. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  4. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  5. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
  6. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,380
  7. 2019Contrastive Multiview Codingcited by 1,301
  8. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,273
  9. 2016Colorful Image Colorizationcited by 1,114
  10. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,066
  11. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,060
  12. 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,028
  13. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 808
  14. 2020Prototypical Contrastive Learning of Unsupervised Representationscited by 640
  15. 2017Large Batch Training of Convolutional Networkscited by 535
  16. 2019A Theoretical Analysis of Contrastive Unsupervised Representation Learningcited by 530
  17. 2019Self-labelling via simultaneous clustering and representation learningcited by 528
  18. 2020Hard Negative Mixing for Contrastive Learningcited by 402
  19. 2020Debiased Contrastive Learningcited by 356
  20. 2019S4L: Self-Supervised Semi-Supervised Learningcited by 335
  21. 2019Large Scale Adversarial Representation Learningcited by 239
  22. 2019Local Aggregation for Unsupervised Learning of Visual Embeddingscited by 204
  23. 2020Boosting Contrastive Self-Supervised Learning with False Negative Cancellationcited by 119
  24. 2019Image inpainting: A reviewcited by 89
  25. 2021ReSSL: Relational Self-Supervised Learning with Weak Augmentationcited by 84
  26. 2020GreedyNAS: Towards Fast One-Shot NAS with Greedy Supernetcited by 69
  27. 2021BCNet: Searching for Network Width with Bilaterally Coupled Networkcited by 23
  28. 2021K-shot NAS: Learnable Weight-Sharing for NAS with K-shot Supernetscited by 22
  29. 2019Contrastive Predictive Coding Based Feature for Automatic Speaker Verificationcited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 19; 2023: 25; 2024: 16; 2025: 13; 2026: 4.

81 papers.

Sort:

By area:

  1. 2022MedCLIP: Contrastive Learning from Unpaired Medical Images and Textcited by 570
  2. 2022SimMatch: Semi-supervised Learning with Similarity Matchingcited by 118
  3. 2023Dynamic Conceptional Contrastive Learning for Generalized Category Discoverycited by 92
  4. 2022PromptCAL: Contrastive Affinity Learning via Auxiliary Prompts for Generalized Novel Category Discoverycited by 87
  5. 2021Learning Representation for Clustering via Prototype Scattering and Positive Samplingcited by 58
  6. 2022Green Hierarchical Vision Transformer for Masked Image Modelingcited by 50
  7. 2023Learning Audio-Visual Source Localization via False Negative Aware Contrastive Learningcited by 47
  8. 2022LightViT: Towards Light-Weight Convolution-Free Vision Transformerscited by 40
  9. 2023InfoNCE Loss Provably Learns Cluster-Preserving Representationscited by 37
  10. 2022OpenCon: Open-world Contrastive Learningcited by 33
  11. 2023Learning Visual Representations via Language-Guided Samplingcited by 32
  12. 2022Learning Where to Learn in Cross-View Self-Supervised Learningcited by 31
38 more
  1. 2023S-CLIP: Semi-supervised Vision-Language Learning using Few Specialist Captionscited by 30
  2. 2024Multi-Label Guided Soft Contrastive Learning for Efficient Earth Observation Pretrainingcited by 28
  3. 2022Multiple Instance Learning via Iterative Self-Paced Supervised Contrastive Learningcited by 22
  4. 2021Learning to Align Sequential Actions in the Wildcited by 20
  5. 2025ProtoGCD: Unified and Unbiased Prototype Learning for Generalized Category Discoverycited by 19
  6. 2021Similarity Contrastive Estimation for Self-Supervised Soft Contrastive Learningcited by 18
  7. 2022Utilizing Expert Features for Contrastive Learning of Time-Series Representationscited by 17
  8. 2023Shrinking Class Space for Enhanced Certainty in Semi-Supervised Learningcited by 17
  9. 2023Unsupervised Object Localization in the Era of Self-Supervised ViTs: A Surveycited by 16
  10. 2024SeBot: Structural Entropy Guided Multi-View Contrastive Learning for Social Bot Detectioncited by 16
  11. 2023OpenForest: A data catalogue for machine learning in forest monitoringcited by 15
  12. 2025Cross-Modal and Uncertainty-Aware Agglomeration for Open-Vocabulary 3D Scene Understandingcited by 13
  13. 2023Weakly-supervised Contrastive Learning for Unsupervised Object Discoverycited by 12
  14. 2023SimMatchV2: Semi-Supervised Learning with Graph Consistencycited by 12
  15. 2024Open-Vocabulary 3D Semantic Segmentation with Text-to-Image Diffusion Modelscited by 12
  16. 2022DyRep: Bootstrapping Training with Dynamic Re-parameterizationcited by 11
  17. 2022Integrating Prior Knowledge in Contrastive Learning with Kernelcited by 11
  18. 2023Contextually Affinitive Neighborhood Refinery for Deep Clusteringcited by 10
  19. 2023A Multi-Modal Contrastive Diffusion Model for Therapeutic Peptide Generationcited by 10
  20. 2024Cluster Aware Graph Anomaly Detectioncited by 8
  21. 2021Deeper-GXX: Deepening Arbitrary GNNscited by 7
  22. 2023Detecting Heart Disease from Multi-View Ultrasound Images via Supervised Attention Multiple Instance Learningcited by 7
  23. 2023BatchSampler: Sampling Mini-Batches for Contrastive Learning in Vision, Language, and Graphscited by 7
  24. 2022Learning Weakly-Supervised Contrastive Representationscited by 6
  25. 2023Fairness-aware Multi-view Clusteringcited by 6
  26. 2023DimCL: Dimensional Contrastive Learning For Improving Self-Supervised Learningcited by 6
  27. 2024Bootstrap Latents of Nodes and Neighbors for Graph Self-Supervised Learningcited by 6
  28. 2024Adversarial Feature Alignment: Balancing Robustness and Accuracy in Deep Learning via Adversarial Trainingcited by 5
  29. 2025Object-aware Sound Source Localization via Audio-Visual Scene Understandingcited by 5
  30. 2023Generalized Category Discovery with Clustering Assignment Consistencycited by 4
  31. 2024Multi-Task Multi-Modal Self-Supervised Learning for Facial Expression Recognitioncited by 4
  32. 2025SuperCL: Superpixel Guided Contrastive Learning for Medical Image Segmentation Pre-trainingcited by 4
  33. 2022Weak Augmentation Guided Relational Self-Supervised Learningcited by 3
  34. 2022Lesion-Aware Contrastive Representation Learning for Histopathology Whole Slide Images Analysiscited by 3
  35. 2022Rethinking Rotation in Self-Supervised Contrastive Learning: Adaptive Positive or Negative Data Augmentationcited by 3
  36. 2023Time-Series Contrastive Learning against False Negatives and Class Imbalancecited by 3
  37. 2024Open-Vocabulary Mobile Manipulation Based on Double Relaxed Contrastive Learning with Dense Labelingcited by 3
  38. 2022Negative Samples are at Large: Leveraging Hard-distance Elastic Loss for Re-identificationcited by 2

50 most cited of 81 All 81 (JSON)