mlsanityData to 7 Oct 2026

When Does Self-supervision Improve Few-shot Learning?

arXiv 1910.03560 ↗First posted 8 Oct 2019cs.CV cs.LGcited by 78

Not on the map. Why?

Abstract (arXiv)

We investigate the role of self-supervised learning (SSL) in the context of few-shot learning. Although recent research has shown the benefits of SSL on large unlabeled datasets, its utility on small datasets is relatively unexplored. We find that SSL reduces the relative error rate of few-shot meta-learners by 4%-27%, even when the datasets are small and only utilizing images within the datasets. The improvements are greater when the training set is smaller or the task is more challenging. Although the benefits of SSL may increase with larger training sets, we observe that SSL can hurt the performance when the distributions of images used for meta-learning and SSL are different. We conduct a systematic study by varying the degree of domain shift and analyzing the performance of several meta-learners on a multitude of domains. Based on this analysis we present a technique that automatically selects images for SSL from a large, generic pool of unlabeled images for a given dataset that provides further improvements.

Key facts. Cited by 78 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 29; 2022: 28; 2023: 12; 2024: 7; 2025: 1; 2026: 1.

78 papers.

Sort:

By area:

  1. 2021Benchmarking Representation Learning for Natural World Image Collectionscited by 167
  2. 2022A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunitiescited by 162
  3. 2021Fine-Grained Image Analysis with Deep Learning: A Surveycited by 135
  4. 2021Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformercited by 116
  5. 2022Improving Test-Time Adaptation via Shift-agnostic Weight Regularization and Nearest Source Prototypescited by 67
  6. 2021MT3: Meta Test-Time Training for Self-Supervised Test-Time Adaptioncited by 65
  7. 2021Exploring Complementary Strengths of Invariant and Equivariant Representations for Few-Shot Learningcited by 56
  8. 2021Factors of Influence for Transfer Learning across Diverse Appearance Domains and Task Typescited by 54
  9. 2021Pareto Self-Supervised Training for Few-Shot Learningcited by 52
  10. 2021A Realistic Evaluation of Semi-Supervised Learning for Fine-Grained Classificationcited by 36
  11. 2022Rethinking Generalization in Few-Shot Classificationcited by 36
  12. 2022Few-Shot Classification with Contrastive Learningcited by 31
38 more
  1. 2021On Fast Adversarial Robustness Adaptation in Model-Agnostic Meta-Learningcited by 29
  2. 2022Matching Feature Sets for Few-Shot Image Classificationcited by 25
  3. 2021LibFewShot: A Comprehensive Library for Few-shot Learningcited by 21
  4. 2022Learning from Future: A Novel Self-Training Framework for Semantic Segmentationcited by 21
  5. 2022Self-Supervision Can Be a Good Few-Shot Learnercited by 19
  6. 2021Reinforced Attention for Few-Shot Learning and Beyondcited by 18
  7. 2022How does unlabeled data improve generalization in self-training? A one-hidden-layer theoretical analysiscited by 18
  8. 2021Contrastive Prototype Learning with Augmented Embeddings for Few-Shot Learningcited by 15
  9. 2021Rich Semantics Improve Few-shot Learningcited by 15
  10. 2021Adaptive Poincar'e Point to Set Distance for Few-Shot Classificationcited by 15
  11. 2022POODLE: Improving Few-shot Learning via Penalizing Out-of-Distribution Samplescited by 15
  12. 2021On the Importance of Distractors for Few-Shot Classificationcited by 14
  13. 2022Tree Structure-Aware Few-Shot Image Classification via Hierarchical Aggregationcited by 14
  14. 2022Task Agnostic Representation Consolidation: a Self-supervised based Continual Learning Approachcited by 13
  15. 2022Multi-level Second-order Few-shot Learningcited by 12
  16. 2021Semi-Supervised Learning with Taxonomic Labelscited by 11
  17. 2021Revisiting Local Descriptor for Improved Few-Shot Classificationcited by 8
  18. 2023GenCo: An Auxiliary Generator from Contrastive Learning for Enhanced Few-Shot Learning in Remote Sensingcited by 8
  19. 2023GeNIe: Generative Hard Negative Images Through Diffusioncited by 8
  20. 2024A Survey of the Self Supervised Learning Mechanisms for Vision Transformerscited by 8
  21. 2021Unsupervised Embedding Adaptation via Early-Stage Feature Reconstruction for Few-Shot Classificationcited by 7
  22. 2022RankDNN: Learning to Rank for Few-shot Learningcited by 7
  23. 2023Pulling Target to Source: A New Perspective on Domain Adaptive Semantic Segmentationcited by 7
  24. 2021Supervised Momentum Contrastive Learning for Few-Shot Classificationcited by 6
  25. 2021Rethinking Nearest Neighbors for Visual Classificationcited by 6
  26. 2022tSF: Transformer-based Semantic Filter for Few-Shot Learningcited by 6
  27. 2023ESPT: A Self-Supervised Episodic Spatial Pretext Task for Improving Few-Shot Learningcited by 6
  28. 2021CSS-LM: A Contrastive Framework for Semi-supervised Fine-tuning of Pre-trained Language Modelscited by 5
  29. 2022Self-Supervised Graph Neural Network for Multi-Source Domain Adaptationcited by 5
  30. 2023Instance-based Max-margin for Practical Few-shot Recognitioncited by 5
  31. 2023PARTICLE: Part Discovery and Contrastive Learning for Fine-grained Recognitioncited by 5
  32. 2024MetaCoCo: A New Few-Shot Classification Benchmark with Spurious Correlationcited by 5
  33. 2022How Well Do Self-Supervised Methods Perform in Cross-Domain Few-Shot Learning?cited by 4
  34. 2022PatchMix Augmentation to Identify Causal Features in Few-shot Learningcited by 4
  35. 2023Clustered-patch Element Connection for Few-shot Learningcited by 4
  36. 2026MMEarth-Bench: Global Model Adaptation via Multimodal Test-Time Trainingcited by 4
  37. 2021Incremental Meta-Learning via Episodic Replay Distillation for Few-Shot Image Recognitioncited by 3
  38. 2022Query-Guided Networks for Few-shot Fine-grained Classification and Person Searchcited by 3

50 most cited of 78 All 78 (JSON)