mlsanityData to 7 Oct 2026

Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with Support Samples

arXiv 2104.13963 ↗First posted 28 Apr 2021cs.CV cs.AI cs.LGcited by 114

Not on the map. Why?

Abstract (arXiv)

This paper proposes a novel method of learning by predicting view assignments with support samples (PAWS). The method trains a model to minimize a consistency loss, which ensures that different views of the same unlabeled instance are assigned similar pseudo-labels. The pseudo-labels are generated non-parametrically, by comparing the representations of the image views to those of a set of randomly sampled labeled images. The distance between the view representations and labeled representations is used to provide a weighting over class labels, which we interpret as a soft pseudo-label. By non-parametrically incorporating labeled samples in this way, PAWS extends the distance-metric loss used in self-supervised methods such as BYOL and SwAV to the semi-supervised setting. Despite the simplicity of the approach, PAWS outperforms other semi-supervised methods across architectures, setting a new state-of-the-art for a ResNet-50 on ImageNet trained with either 10% or 1% of the labels, reaching 75.5% and 66.5% top-1 respectively. PAWS requires 4x to 12x less training than the previous best methods.

Key facts. Cited by 114 papers in the map (data through 7 Oct 2026). Builds on 34 of its 56 references that are in the map. First posted on arXiv 28 Apr 2021.

Builds on

34 of 56 references.

References · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  5. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  6. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  8. 2016Wide Residual Networkscited by 3,057
  9. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  10. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  11. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  12. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
22 more
  1. 2020Supervised Contrastive Learningcited by 2,803
  2. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  3. 2016Matching Networks for One Shot Learningcited by 2,246
  4. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
  5. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,390
  6. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  7. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
  8. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,054
  9. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 918
  10. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 806
  11. 2017Large Batch Training of Convolutional Networkscited by 529
  12. 2019Self-labelling via simultaneous clustering and representation learningcited by 525
  13. 2020Meta Pseudo Labelscited by 372
  14. 2020Rethinking Pre-training and Self-trainingcited by 352
  15. 2019ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoringcited by 349
  16. 2019S4L: Self-Supervised Semi-Supervised Learningcited by 335
  17. 2019Interpolation Consistency Training for Semi-Supervised Learningcited by 248
  18. 2019MultiGrain: a unified image embedding for classes and instancescited by 85
  19. 2018Unsupervised Learning via Meta-Learningcited by 81
  20. 2019EnAET: A Self-Trained framework for Semi-Supervised and Supervised Learning with Ensemble Transformationscited by 11
  21. 2019Semi-Supervised Learning by Label Gradient Alignmentcited by 4
  22. 2020Supervision Accelerates Pre-training in Contrastive Semi-Supervised Learning of Visual Representationscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 32; 2023: 33; 2024: 28; 2025: 10; 2026: 3.

114 papers.

Sort:

By area:

  1. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 982
  2. 2021Understanding Dimensional Collapse in Contrastive Self-supervised Learningcited by 374
  3. 2022Masked Siamese Networks for Label-Efficient Learningcited by 301
  4. 2021Sustainable AI: Environmental Implications, Challenges and Opportunitiescited by 242
  5. 2022How to Exploit Hyperspherical Embeddings for Out-of-Distribution Detection?cited by 128
  6. 2022SimMatch: Semi-supervised Learning with Similarity Matchingcited by 118
  7. 2021Are Large-scale Datasets Necessary for Self-Supervised Pre-training?cited by 117
  8. 2022Parametric Classification for Generalized Category Discovery: A Baseline Studycited by 113
  9. 2022Debiased Learning from Naturally Imbalanced Pseudo-Labelscited by 85
  10. 2022Pushing the limits of self-supervised ResNets: Can we outperform supervised learning without labels on ImageNet?cited by 67
  11. 2022Improving Test-Time Adaptation via Shift-agnostic Weight Regularization and Nearest Source Prototypescited by 67
  12. 2022Debiased Self-Training for Semi-Supervised Learningcited by 62
38 more
  1. 2023No Representation Rules Them All in Category Discoverycited by 61
  2. 2023Learning Customized Visual Models with Retrieval-Augmented Knowledgecited by 58
  3. 2022The Hidden Uniform Cluster Prior in Self-Supervised Learningcited by 57
  4. 2022Semi-supervised Vision Transformers at Scalecited by 49
  5. 2023STUNT: Few-shot Tabular Learning with Self-generated Tasks from Unlabeled Tablescited by 48
  6. 2023CoCo: A Coupled Contrastive Framework for Unsupervised Domain Adaptive Graph Classificationcited by 46
  7. 2023LaFTer: Label-Free Tuning of Zero-shot Classifier using Language and Unlabeled Image Collectionscited by 43
  8. 2024Learning with Mixture of Prototypes for Out-of-Distribution Detectioncited by 43
  9. 2023Towards Open-Set Test-Time Adaptation Utilizing the Wisdom of Crowds in Entropy Minimizationcited by 39
  10. 2023Diffusion Models and Semi-Supervised Learners Benefit Mutually with Few Labelscited by 37
  11. 2022A Closer Look at Self-Supervised Lightweight Vision Transformerscited by 31
  12. 2024Exemplar-free Continual Representation Learning via Learnable Drift Compensationcited by 30
  13. 2023S-CLIP: Semi-supervised Vision-Language Learning using Few Specialist Captionscited by 29
  14. 2021Unsupervised Selective Labeling for More Effective Semi-Supervised Learningcited by 27
  15. 2023CiPR: An Efficient Framework with Cross-instance Positive Relations for Generalized Category Discoverycited by 26
  16. 2022Non-Contrastive Learning Meets Language-Image Pre-Trainingcited by 25
  17. 2022DeepCut: Unsupervised Segmentation using Graph Neural Networks Clusteringcited by 25
  18. 2023Incremental Generalized Category Discoverycited by 25
  19. 2022No Reason for No Supervision: Improved Generalization in Supervised Modelscited by 23
  20. 2022Masked Siamese ConvNetscited by 22
  21. 2022Prompt Tuning for Parameter-efficient Medical Image Segmentationcited by 21
  22. 2022Location-Aware Self-Supervised Transformers for Semantic Segmentationcited by 20
  23. 2023Improved baselines for vision-language pre-trainingcited by 20
  24. 2023Semi-supervised learning made simple with self-supervised clusteringcited by 20
  25. 2022A soft nearest-neighbor framework for continual semi-supervised learningcited by 17
  26. 2023Shrinking Class Space for Enhanced Certainty in Semi-Supervised Learningcited by 17
  27. 2023TimeBalance: Temporally-Invariant and Temporally-Distinctive Video Representations for Semi-Supervised Action Recognitioncited by 16
  28. 2022MemSAC: Memory Augmented Sample Consistency for Large Scale Unsupervised Domain Adaptationcited by 15
  29. 2023Plasticity-Optimized Complementary Networks for Unsupervised Continual Learningcited by 14
  30. 2024DINO as a von Mises-Fisher mixture modelcited by 14
  31. 2024Effective Prompt Pool Learning for Continual Category Discoverycited by 14
  32. 2023ProtoCon: Pseudo-label Refinement via Online Clustering and Prototypical Consistency for Efficient Semi-supervised Learningcited by 13
  33. 2023SimMatchV2: Semi-Supervised Learning with Graph Consistencycited by 12
  34. 2024Denoising with a Joint-Embedding Predictive Architecturecited by 12
  35. 2023DeDrift: Robust Similarity Search under Content Driftcited by 11
  36. 2022GraFN: Semi-Supervised Node Classification on Graph with Few Labels via Non-Parametric Distribution Assignmentcited by 10
  37. 2023RoPAWS: Robust Semi-supervised Representation Learning from Uncurated Datacited by 10
  38. 2024Tabular Transfer Learning via Prompting LLMscited by 10

50 most cited of 114 All 114 (JSON)