mlsanityData to 14 Oct 2026

Visual-Semantic Contrastive Alignment for Few-Shot Image Classification

arXiv 2210.11000 ↗First posted 20 Oct 2022cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Few-Shot learning aims to train and optimize a model that can adapt to unseen visual classes with only a few labeled examples. The existing few-shot learning (FSL) methods, heavily rely only on visual data, thus fail to capture the semantic attributes to learn a more generalized version of the visual concept from very few examples. However, it is a known fact that human visual learning benefits immensely from inputs from multiple modalities such as vision, language, and audio. Inspired by the human learning nature of encapsulating the existing knowledge of a visual category which is in the form of language, we introduce a contrastive alignment mechanism for visual and semantic feature vectors to learn much more generalized visual concepts for few-shot learning. Our method simply adds an auxiliary contrastive learning objective which captures the contextual knowledge of a visual category from a strong textual encoder in addition to the existing training mechanism. Hence, the approach is more generalized and can be plugged into any existing FSL method. The pre-trained semantic feature extractor (learned from a large-scale text corpora) we use in our approach provides a strong contextual prior knowledge to assist FSL. The experimental results done in popular FSL datasets show that our approach is generic in nature and provides a strong boost to the existing FSL baselines.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 22 of its 24 references that are in the map. First posted on arXiv 20 Oct 2022.

Builds on

22 of 24 references.

References · 22

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  6. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  7. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  8. 2016Matching Networks for One Shot Learningcited by 2,255
  9. 2019A Closer Look at Few-shot Classificationcited by 684
  10. 2020Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?cited by 437
  11. 2022CrossPoint: Self-Supervised Cross-Modal Contrastive Learning for 3D Point Cloud Understandingcited by 198
  12. 2020Audio-Visual Instance Discrimination with Cross-Modal Agreementcited by 195
10 more
  1. 2016Learning Deep Representations of Fine-grained Visual Descriptionscited by 182
  2. 2020Meta-Baseline: Exploring Simple Meta-Learning for Few-Shot Learningcited by 179
  3. 2019Boosting Few-Shot Visual Learning with Self-Supervisioncited by 140
  4. 2019Adaptive Cross-Modal Few-Shot Learningcited by 95
  5. 2019When Does Self-supervision Improve Few-shot Learning?cited by 78
  6. 2017Learning with Latent Languagecited by 75
  7. 2020Self-supervised Knowledge Distillation for Few-shot Learningcited by 41
  8. 2019Baby steps towards few-shot learning with multiple semanticscited by 29
  9. 2019Shaping Visual Representations with Language for Few-shot Classificationcited by 27
  10. 2021Rich Semantics Improve Few-shot Learningcited by 15

Built on by

Cited by year

Citing papers per year: 2023: 3.

3 papers.

  1. 2023FILM: How can Few-Shot Image Classification Benefit from Pre-Trained Language Models?cited by 2
  2. 2023SMC-NCA: Semantic-guided Multi-level Contrast for Semi-supervised Temporal Action Segmentationcited by 1
  3. 2023TransMed: Large Language Models Enhance Vision Transformer for Biomedical Image Classificationcited by 0