mlsanityData to 14 Oct 2026

Visual Representation Learning with Self-Supervised Attention for Low-Label High-data Regime

arXiv 2201.08951 ↗First posted 22 Jan 2022cs.CV cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Self-supervision has shown outstanding results for natural language processing, and more recently, for image recognition. Simultaneously, vision transformers and its variants have emerged as a promising and scalable alternative to convolutions on various computer vision tasks. In this paper, we are the first to question if self-supervised vision transformers (SSL-ViTs) can be adapted to two important computer vision tasks in the low-label, high-data regime: few-shot image classification and zero-shot image retrieval. The motivation is to reduce the number of manual annotations required to train a visual embedder, and to produce generalizable and semantically meaningful embeddings. For few-shot image classification we train SSL-ViTs without any supervision, on external data, and use this trained embedder to adapt quickly to novel classes with limited number of labels. For zero-shot image retrieval, we use SSL-ViTs pre-trained on a large dataset without any labels and fine-tune them with several metric learning objectives. Our self-supervised attention representations outperforms the state-of-the-art on several public benchmarks for both tasks, namely miniImageNet and CUB200 for few-shot image classification by up-to 6%-10%, and Stanford Online Products, Cars196 and CUB200 for zero-shot image retrieval by up-to 4%-11%. Code is available at urlhttps://github.com/AutoVision-cloud/SSL-ViT-lowlabel-highdata.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 24 references that are in the map. First posted on arXiv 22 Jan 2022.

Builds on

21 of 24 references.

References · 21

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  2. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  3. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  5. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  6. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  7. 2015Deep Metric Learning via Lifted Structured Feature Embeddingcited by 449
  8. 2017Meta-SGD: Learning to Learn Quickly for Few-Shot Learningcited by 379
  9. 2019Multi-Similarity Loss with General Pair Weighting for Deep Metric Learningcited by 368
  10. 2020A Metric Learning Reality Checkcited by 321
  11. 2017Sampling Matters in Deep Embedding Learningcited by 285
  12. 2017No Fuss Distance Metric Learning using Proxiescited by 226
9 more
  1. 2020CrossTransformers: spatially-aware few-shot transfercited by 174
  2. 2021Free Lunch for Few-shot Learning: Distribution Calibrationcited by 165
  3. 2019Boosting Few-Shot Visual Learning with Self-Supervisioncited by 140
  4. 2020Negative Margin Matters: Understanding Margin in Few-shot Classificationcited by 129
  5. 2019When Does Self-supervision Improve Few-shot Learning?cited by 78
  6. 2018Unsupervised Meta-Learning For Few-Shot Image Classificationcited by 75
  7. 2018Multi-level Semantic Feature Augmentation for One-shot Learningcited by 62
  8. 2019Self-Supervised Learning For Few-Shot Image Classificationcited by 44
  9. 2021Towards Interpretable Deep Metric Learning with Structural Matchingcited by 30

Built on by

Not cited yet.