# Self-Supervised Prototypical Transfer Learning for Few-Shot Classification

arXiv: https://arxiv.org/abs/2006.11325 · HTML: https://mlsanity.com/paper/2006.11325/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Jun 2020.

## Built on by (most cited first, 26 of 26)

- [Exploring Complementary Strengths of Invariant and Equivariant Representations for Few-Shot Learning](https://mlsanity.com/paper/2103.01315/) (2021)
- [Self-Supervision Can Be a Good Few-Shot Learner](https://mlsanity.com/paper/2207.09176/) (2022)
- [Contrastive Prototype Learning with Augmented Embeddings for Few-Shot Learning](https://mlsanity.com/paper/2101.09499/) (2021)
- [Bridge to Target Domain by Prototypical Contrastive Learning and Label Confusion: Re-explore Zero-Shot Learning for Slot Filling](https://mlsanity.com/paper/2110.03572/) (2021)
- [Unsupervised Few-Shot Action Recognition via Action-Appearance Aligned Meta-Adaptation](https://mlsanity.com/paper/2109.15317/) (2021)
- [A Survey of the Self Supervised Learning Mechanisms for Vision Transformers](https://mlsanity.com/paper/2408.17059/) (2024)
- [ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback](https://mlsanity.com/paper/2107.14035/) (2021)
- [Recent Few-Shot Object Detection Algorithms: A Survey with Performance Comparison](https://mlsanity.com/paper/2203.14205/) (2022)
- [GeNIe: Generative Hard Negative Images Through Diffusion](https://mlsanity.com/paper/2312.02548/) (2023)
- [tSF: Transformer-based Semantic Filter for Few-Shot Learning](https://mlsanity.com/paper/2211.00868/) (2022)
- [BECLR: Batch Enhanced Contrastive Few-Shot Learning](https://mlsanity.com/paper/2402.02444/) (2024)
- [Empirical Evaluation and Theoretical Analysis for Representation Learning: A Survey](https://mlsanity.com/paper/2204.08226/) (2022)
- [Online Unsupervised Learning of Visual Representations and Categories](https://mlsanity.com/paper/2109.05675/) (2021)
- [How Well Do Self-Supervised Methods Perform in Cross-Domain Few-Shot Learning?](https://mlsanity.com/paper/2202.09014/) (2022)
- [Self-Attention Message Passing for Contrastive Few-Shot Learning](https://mlsanity.com/paper/2210.06339/) (2022)
- [Meta-DM: Applications of Diffusion Models on Few-Shot Learning](https://mlsanity.com/paper/2305.08092/) (2023)
- [Dynamic Sub-Cluster-Aware Network for Few-Shot Skin Disease Classification](https://mlsanity.com/paper/2207.01072/) (2022)
- [Rethinking Clustering-Based Pseudo-Labeling for Unsupervised Meta-Learning](https://mlsanity.com/paper/2209.13635/) (2022)
- [Rethinking the Metric in Few-shot Learning: From an Adaptive Multi-Distance Perspective](https://mlsanity.com/paper/2211.00890/) (2022)
- [A Survey of Deep Visual Cross-Domain Few-Shot Learning](https://mlsanity.com/paper/2303.09253/) (2023)
- [Self-Supervised Class-Cognizant Few-Shot Classification](https://mlsanity.com/paper/2202.08149/) (2022)
- [DeepFT: Fault-Tolerant Edge Computing using a Self-Supervised Deep Surrogate Model](https://mlsanity.com/paper/2212.01302/) (2022)
- [Few-Shot Electronic Health Record Coding through Graph Contrastive Learning](https://mlsanity.com/paper/2106.15467/) (2021)
- [Cross-Language Speech Emotion Recognition Using Multimodal Dual Attention Transformers](https://mlsanity.com/paper/2306.13804/) (2023)
- [Boosting Few-Shot Learning with Disentangled Self-Supervised Learning and Meta-Learning for Medical Image Classification](https://mlsanity.com/paper/2403.17530/) (2024)
- [BatMan-CLR: Making Few-shots Meta-Learners Resilient Against Label Noise](https://mlsanity.com/paper/2309.06046/) (2023)

## Abstract (arXiv)

Most approaches in few-shot learning rely on costly annotated data related to the goal task domain during (pre-)training. Recently, unsupervised meta-learning methods have exchanged the annotation requirement for a reduction in few-shot classification performance. Simultaneously, in settings with realistic domain shift, common transfer learning has been shown to outperform supervised meta-learning. Building on these insights and on advances in self-supervised learning, we propose a transfer learning approach which constructs a metric embedding that clusters unlabeled prototypical samples and their augmentations closely together. This pre-trained embedding is a starting point for few-shot classification by summarizing class clusters and fine-tuning. We demonstrate that our self-supervised prototypical transfer learning approach ProtoTransfer outperforms state-of-the-art unsupervised meta-learning methods on few-shot tasks from the mini-ImageNet dataset. In few-shot experiments with domain shift, our approach even has comparable performance to supervised methods, but requires orders of magnitude fewer labels.