Improving Few-Shot Learning with Auxiliary Self-Supervised Pretext Tasks
Not on the map. Why?
Abstract (arXiv)
Recent work on few-shot learning citetian2020rethinking showed that quality of learned representations plays an important role in few-shot classification performance. On the other hand, the goal of self-supervised learning is to recover useful semantic information of the data without the use of class labels. In this work, we exploit the complementarity of both paradigms via a multi-task framework where we leverage recent self-supervised methods as auxiliary tasks. We found that combining multiple tasks is often beneficial, and that solving them simultaneously can be done efficiently. Our results suggest that self-supervised auxiliary tasks are effective data-dependent regularizers for representation learning. Our code is available at: urlhttps://github.com/nathanielsimard/improving-fs-ssl.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 42 of its 53 references that are in the map. First posted on arXiv 24 Jan 2021.
Builds on
42 of 53 references.
References · 42
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
- 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
- 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
- 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
- 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
- 2017Prototypical Networks for Few-shot Learningcited by 3,015
- 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
- 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
30 more
- 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
- 2016Matching Networks for One Shot Learningcited by 2,255
- 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
- 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,628
- 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
- 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
- 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,380
- 2019Contrastive Multiview Codingcited by 1,301
- 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,143
- 2016Colorful Image Colorizationcited by 1,114
- 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,066
- 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 808
- 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 734
- 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 673
- 2018Meta-Learning with Latent Embedding Optimizationcited by 476
- 2019Meta-Learning with Differentiable Convex Optimizationcited by 461
- 2020Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?cited by 437
- 2018TADAM: Task dependent adaptive metric for improved few-shot learningcited by 408
- 2018Meta-learning with differentiable closed-form solverscited by 397
- 2018Dynamic Few-Shot Visual Learning without Forgettingcited by 385
- 2019Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAMLcited by 359
- 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 318
- 2019A Baseline for Few-Shot Image Classificationcited by 298
- 2018DropBlock: A regularization method for convolutional networkscited by 262
- 2016Learning Representations for Automatic Colorizationcited by 251
- 2019Kornia: an Open Source Differentiable Computer Vision Library for PyTorchcited by 232
- 2017Multi-task Self-Supervised Visual Learningcited by 200
- 2019Boosting Few-Shot Visual Learning with Self-Supervisioncited by 140
- 2019When Does Self-supervision Improve Few-shot Learning?cited by 78
- 2020Understanding Self-supervised Learning with Dual Deep Networkscited by 62
Built on by
Citing papers per year: 2023: 1.
1 paper.