LPN: Language-guided Prototypical Network for few-shot classification
Not on the map. Why?
Abstract (arXiv)
Few-shot classification aims to adapt to new tasks with limited labeled examples. To fully use the accessible data, recent methods explore suitable measures for the similarity between the query and support images and better high-dimensional features with meta-training and pre-training strategies. However, the potential of multi-modality information has barely been explored, which may bring promising improvement for few-shot classification. In this paper, we propose a Language-guided Prototypical Network (LPN) for few-shot classification, which leverages the complementarity of vision and language modalities via two parallel branches to improve the classifier. Concretely, to introduce language modality with limited samples in the visual task, we leverage a pre-trained text encoder to extract class-level text features directly from class names while processing images with a conventional image encoder. Then, we introduce a language-guided decoder to obtain text features corresponding to each image by aligning class-level features with visual features. Additionally, we utilize class-level features and prototypes to build a refined prototypical head, which generates robust prototypes in the text branch for follow-up measurement. Furthermore, we leverage the class-level features to align the visual features, capturing more class-relevant visual features. Finally, we aggregate the visual and text logits to calibrate the deviation of a single modality, enhancing the overall performance. Extensive experiments demonstrate the competitiveness of LPN against state-of-the-art methods on benchmark datasets.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 44 of its 62 references that are in the map. First posted on arXiv 4 Jul 2023.
Builds on
44 of 62 references.
References · 44
- 2017Attention Is All You Needcited by 59,932
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
- 2022Generative Adversarial Networkscited by 5,603
- 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
- 2017Prototypical Networks for Few-shot Learningcited by 3,001
- 2015Fast R-CNNcited by 2,930
- 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
- 2021Learning to Prompt for Vision-Language Modelscited by 2,269
32 more
- 2016Matching Networks for One Shot Learningcited by 2,246
- 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,139
- 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,021
- 2019A Closer Look at Few-shot Classificationcited by 679
- 2021DenseCLIP: Language-Guided Dense Prediction with Context-Aware Promptingcited by 530
- 2018Meta-Learning with Latent Embedding Optimizationcited by 476
- 2019Meta-Learning with Differentiable Convex Optimizationcited by 461
- 2018Meta-Learning for Semi-Supervised Few-Shot Classificationcited by 456
- 2020Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?cited by 435
- 2021Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modelingcited by 429
- 2018TADAM: Task dependent adaptive metric for improved few-shot learningcited by 408
- 2019Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAMLcited by 359
- 2021MagFace: A Universal Representation for Face Recognition and Quality Assessmentcited by 332
- 2018Few-Shot Learning via Embedding Adaptation with Set-to-Set Functionscited by 293
- 2020Meta-Baseline: Exploring Simple Meta-Learning for Few-Shot Learningcited by 179
- 2019Charting the Right Manifold: Manifold Mixup for Few-shot Learningcited by 135
- 2020Negative Margin Matters: Understanding Margin in Few-shot Classificationcited by 129
- 2019Revisiting Local Descriptor based Image-to-Class Measure for Few-shot Learningcited by 121
- 2020Few-Shot Classification with Feature Map Reconstruction Networkscited by 116
- 2019Finding Task-Relevant Features for Few-Shot Learning by Category Traversalcited by 101
- 2019Adaptive Cross-Modal Few-Shot Learningcited by 95
- 2020Laplacian Regularized Few-Shot Learningcited by 90
- 2019Associative Alignment for Few-shot Image Classificationcited by 58
- 2020Multimodal Prototypical Networks for Few-shot Learningcited by 48
- 2020Self-supervised Knowledge Distillation for Few-shot Learningcited by 41
- 2019Baby steps towards few-shot learning with multiple semanticscited by 29
- 2021Face.evoLVe: A High-Performance Face Recognition Librarycited by 26
- 2021Binocular Mutual Learning for Improving Few-shot Classificationcited by 26
- 2022Few-Shot Learning Meets Transformer: Unified Query-Support Transformers for Few-Shot Classificationcited by 16
- 2021SEGA: Semantic Guided Attention on Visual Prototype for Few-Shot Learningcited by 15
- 2021Dizygotic Conditional Variational AutoEncoder for Multi-Modal and Partial Modality Absent Few-Shot Learningcited by 4
- 2020Meta-Generating Deep Attentive Metric for Few-shot Classificationcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.