# Semantic Prompt for Few-Shot Image Recognition

arXiv: https://arxiv.org/abs/2303.14123 · HTML: https://mlsanity.com/paper/2303.14123/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 61 references that are in the map.
- First posted on arXiv 24 Mar 2023.

## Builds on (most cited first, 30 of 43)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [Big Transfer (BiT): General Visual Representation Learning](https://mlsanity.com/paper/1912.11370/) (2019)
- [Multimodal Few-Shot Learning with Frozen Language Models](https://mlsanity.com/paper/2106.13884/) (2021)
- [Meta-Learning with Latent Embedding Optimization](https://mlsanity.com/paper/1807.05960/) (2018)
- [Meta-Learning with Differentiable Convex Optimization](https://mlsanity.com/paper/1904.03758/) (2019)
- [Meta-Learning for Semi-Supervised Few-Shot Classification](https://mlsanity.com/paper/1803.00676/) (2018)
- [Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?](https://mlsanity.com/paper/2003.11539/) (2020)
- [TADAM: Task dependent adaptive metric for improved few-shot learning](https://mlsanity.com/paper/1805.10123/) (2018)
- [Meta-Baseline: Exploring Simple Meta-Learning for Few-Shot Learning](https://mlsanity.com/paper/2003.04390/) (2020)
- [Boosting Few-Shot Visual Learning with Self-Supervision](https://mlsanity.com/paper/1906.05186/) (2019)
- [Visformer: The Vision-friendly Transformer](https://mlsanity.com/paper/2104.12533/) (2021)
- [Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2108.13161/) (2021)
- [Inception Transformer](https://mlsanity.com/paper/2205.12956/) (2022)
- [Nested Hierarchical Transformer: Towards Accurate, Data-Efficient and Interpretable Visual Understanding](https://mlsanity.com/paper/2105.12723/) (2021)

## Built on by (most cited first, 29 of 29)

- [PromptAD: Learning Prompts with only Normal Samples for Few-Shot Anomaly Detection](https://mlsanity.com/paper/2404.05231/) (2024)
- [MVP-Shot: Multi-Velocity Progressive-Alignment Framework for Few-Shot Action Recognition](https://mlsanity.com/paper/2405.02077/) (2024)
- [Simple Semantic-Aided Few-Shot Learning](https://mlsanity.com/paper/2311.18649/) (2023)
- [TAI++: Text as Image for Multi-Label Image Classification by Co-Learning Transferable Prompt](https://mlsanity.com/paper/2405.06926/) (2024)
- [Pathology-knowledge Enhanced Multi-instance Prompt Learning for Few-shot Whole Slide Image Classification](https://mlsanity.com/paper/2407.10814/) (2024)
- [Prompt-and-Transfer: Dynamic Class-aware Enhancement for Few-shot Segmentation](https://mlsanity.com/paper/2409.10389/) (2024)
- [Connecting Giants: Synergistic Knowledge Transfer of Large Multimodal Models for Few-Shot Learning](https://mlsanity.com/paper/2510.11115/) (2025)
- [Envisioning Class Entity Reasoning by Large Language Models for Few-shot Learning](https://mlsanity.com/paper/2408.12469/) (2024)
- [Making Large Vision Language Models to be Good Few-shot Learners](https://mlsanity.com/paper/2408.11297/) (2024)
- [VT-FSL: Bridging Vision and Text with LLMs for Few-Shot Learning](https://mlsanity.com/paper/2509.25033/) (2025)
- [Visual and Semantic Prompt Collaboration for Generalized Zero-Shot Learning](https://mlsanity.com/paper/2503.23030/) (2025)
- [Spatio-temporal Decoupled Knowledge Compensator for Few-Shot Action Recognition](https://mlsanity.com/paper/2602.18043/) (2026)
- [ConSept: Continual Semantic Segmentation via Adapter-based Vision Transformer](https://mlsanity.com/paper/2402.16674/) (2024)
- [Few-Shot Vision-Language Action-Incremental Policy Learning](https://mlsanity.com/paper/2504.15517/) (2025)
- [FILM: How can Few-Shot Image Classification Benefit from Pre-Trained Language Models?](https://mlsanity.com/paper/2307.04114/) (2023)
- [Embedding Generalized Semantic Knowledge into Few-Shot Remote Sensing Segmentation](https://mlsanity.com/paper/2405.13686/) (2024)
- [Verbalized Representation Learning for Interpretable Few-Shot Generalization](https://mlsanity.com/paper/2411.18651/) (2024)
- [DVLA-RL: Dual-Level Vision-Language Alignment with Reinforcement Learning Gating for Few-Shot Learning](https://mlsanity.com/paper/2602.00795/) (2026)
- [SEM-CLIP: Precise Few-Shot Learning for Nanoscale Defect Detection in Scanning Electron Microscope Image](https://mlsanity.com/paper/2502.14884/) (2025)
- [EfficientFSL: Enhancing Few-Shot Classification via Query-Only Tuning in Vision Transformers](https://mlsanity.com/paper/2601.08499/) (2026)
- [MPA: Multimodal Prototype Augmentation for Few-Shot Learning](https://mlsanity.com/paper/2602.10143/) (2026)
- [Rethinking the Good Enough Embedding for Easy Few-Shot Learning](https://mlsanity.com/paper/2605.14145/) (2026)
- [Language-Guided Reinforcement Learning for Hard Attention in Few-Shot Learning](https://mlsanity.com/paper/2310.07800/) (2023)
- [TransMed: Large Language Models Enhance Vision Transformer for Biomedical Image Classification](https://mlsanity.com/paper/2312.07125/) (2023)
- [RESTORE: Towards Feature Shift for Vision-Language Prompt Learning](https://mlsanity.com/paper/2403.06136/) (2024)
- [Like Humans to Few-Shot Learning through Knowledge Permeation of Vision and Text](https://mlsanity.com/paper/2405.12543/) (2024)
- [MetaLab: Few-Shot Game Changer for Image Recognition](https://mlsanity.com/paper/2507.22057/) (2025)
- [Color as the Impetus: Transforming Few-Shot Learner](https://mlsanity.com/paper/2507.22136/) (2025)
- [PMCE: Probabilistic Multi-Granularity Semantics with Caption-Guided Enhancement for Few-Shot Learning](https://mlsanity.com/paper/2601.14111/) (2026)

## Abstract (arXiv)

Few-shot learning is a challenging problem since only a few examples are provided to recognize a new class. Several recent studies exploit additional semantic information, e.g. text embeddings of class names, to address the issue of rare samples through combining semantic prototypes with visual prototypes. However, these methods still suffer from the spurious visual features learned from the rare support samples, resulting in limited benefits. In this paper, we propose a novel Semantic Prompt (SP) approach for few-shot learning. Instead of the naive exploitation of semantic information for remedying classifiers, we explore leveraging semantic information as prompts to tune the visual feature extraction network adaptively. Specifically, we design two complementary mechanisms to insert semantic prompts into the feature extractor: one is to enable the interaction between semantic prompts and patch embeddings along the spatial dimension via self-attention, another is to supplement visual features with the transformed semantic prompts along the channel dimension. By combining these two mechanisms, the feature extractor presents a better ability to attend to the class-specific features and obtains more generalized image representations with merely a few support samples. Through extensive experiments on four datasets, the proposed approach achieves promising results, improving the 1-shot learning accuracy by 3.67% on average.