mlsanityData to 14 Oct 2026

Supervised Momentum Contrastive Learning for Few-Shot Classification

arXiv 2101.11058 ↗First posted 26 Jan 2021cs.CV cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Few-shot learning aims to transfer information from one task to enable generalization on novel tasks given a few examples. This information is present both in the domain and the class labels. In this work we investigate the complementary roles of these two sources of information by combining instance-discriminative contrastive learning and supervised learning in a single framework called Supervised Momentum Contrastive learning (SUPMOCO). Our approach avoids a problem observed in supervised learning where information in images not relevant to the task is discarded, which hampers their generalization to novel tasks. We show that (self-supervised) contrastive learning and supervised learning are mutually beneficial, leading to a new state-of-the-art on the META-DATASET - a recently introduced benchmark for few-shot learning. Our method is based on a simple modification of MOCO and scales better than prior work on combining supervised and self-supervised learning. This allows us to easily combine data from multiple domains leading to further improvements.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 47 of its 57 references that are in the map. First posted on arXiv 26 Jan 2021.

Builds on

47 of 57 references.

References · 47

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  6. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  7. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  8. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  9. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  10. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  11. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
  12. 2020Supervised Contrastive Learningcited by 2,829
35 more
  1. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  2. 2016Matching Networks for One Shot Learningcited by 2,255
  3. 2013Describing Textures in the Wildcited by 2,125
  4. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
  5. 2013Fine-Grained Visual Classification of Aircraftcited by 1,726
  6. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  7. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
  8. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,143
  9. 2019A Closer Look at Few-shot Classificationcited by 684
  10. 2018Taskonomy: Disentangling Task Transfer Learningcited by 634
  11. 2017Large Batch Training of Convolutional Networkscited by 535
  12. 2019Meta-Learning with Differentiable Convex Optimizationcited by 461
  13. 2020Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?cited by 437
  14. 2014Deep metric learning using Triplet networkcited by 422
  15. 2018TADAM: Task dependent adaptive metric for improved few-shot learningcited by 408
  16. 2018Meta-learning with differentiable closed-form solverscited by 397
  17. 2018Dynamic Few-Shot Visual Learning without Forgettingcited by 385
  18. 2019Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examplescited by 315
  19. 2019A Baseline for Few-Shot Image Classificationcited by 298
  20. 2019Task2Vec: Task Embedding for Meta-Learningcited by 205
  21. 2019SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learningcited by 195
  22. 2020CrossTransformers: spatially-aware few-shot transfercited by 174
  23. 2019Boosting Few-Shot Visual Learning with Self-Supervisioncited by 140
  24. 2019Fast and Flexible Multi-Task Classification Using Conditional Neural Adaptive Processescited by 131
  25. 2019Transferability and Hardness of Supervised Classification Taskscited by 127
  26. 2019Improved Few-Shot Visual Classificationcited by 115
  27. 2020What makes instance discrimination good for transfer learning?cited by 111
  28. 2017Critical Learning Periods in Deep Neural Networkscited by 79
  29. 2019When Does Self-supervision Improve Few-shot Learning?cited by 78
  30. 2020A Universal Representation Transformer Layer for Few-Shot Image Classificationcited by 65
  31. 2019Few-Shot Learning with Embedded Class Models and Shot-Free Meta Trainingcited by 57
  32. 2017Universal representations:The missing link between faces, text, planktons, and cat breedscited by 45
  33. 2019TAFE-Net: Task-Aware Feature Embeddings for Low Shot Learningcited by 32
  34. 2020Neural Data Server: A Large-Scale Search Engine for Transfer Learning Datacited by 27
  35. 2020Selecting Relevant Features from a Multi-domain Representation for Few-shot Classificationcited by 11

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 2; 2023: 1; 2024: 2.

6 papers.

  1. 2021Generalized Few-Shot Semantic Segmentation: All You Need is Fine-Tuningcited by 8
  2. 2022Learning Dense Object Descriptors from Multiple Views for Low-shot Category Generalizationcited by 6
  3. 2024On the Effectiveness of Supervision in Asymmetric Non-Contrastive Learningcited by 3
  4. 2023Low-shot Object Learning with Mutual Exclusivity Biascited by 2
  5. 2022The Best of Both Worlds: a Framework for Combining Degradation Prediction with High Performance Super-Resolution Networkscited by 0
  6. 2024Few and Fewer: Learning Better from Few Examples Using Fewer Base Classescited by 0