mlsanityData to 14 Oct 2026

KNN Transformer with Pyramid Prompts for Few-Shot Learning

arXiv 2410.10227 ↗First posted 14 Oct 2024cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Few-Shot Learning (FSL) aims to recognize new classes with limited labeled data. Recent studies have attempted to address the challenge of rare samples with textual prompts to modulate visual features. However, they usually struggle to capture complex semantic relationships between textual and visual features. Moreover, vanilla self-attention is heavily affected by useless information in images, severely constraining the potential of semantic priors in FSL due to the confusion of numerous irrelevant tokens during interaction. To address these aforementioned issues, a K-NN Transformer with Pyramid Prompts (KTPP) is proposed to select discriminative information with K-NN Context Attention (KCA) and adaptively modulate visual features with Pyramid Cross-modal Prompts (PCP). First, for each token, the KCA only selects the K most relevant tokens to compute the self-attention matrix and incorporates the mean of all tokens as the context prompt to provide the global context in three cascaded stages. As a result, irrelevant tokens can be progressively suppressed. Secondly, pyramid prompts are introduced in the PCP to emphasize visual features via interactions between text-based class-aware prompts and multi-scale visual features. This allows the ViT to dynamically adjust the importance weights of visual features based on rich semantic information at different scales, making models robust to spatial variations. Finally, augmented visual features and class-aware prompts are interacted via the KCA to extract class-specific features. Consequently, our model further enhances noise-free visual representations via deep cross-modal interactions, extracting generalized visual representation in scenarios with few labeled samples. Extensive experiments on four benchmark datasets demonstrate the effectiveness of our method.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 39 of its 66 references that are in the map. First posted on arXiv 14 Oct 2024.

Builds on

39 of 66 references.

References · 39

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  5. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  6. 2021Learning to Prompt for Vision-Language Modelscited by 2,281
  7. 2016Matching Networks for One Shot Learningcited by 2,255
  8. 2021ViViT: A Video Vision Transformercited by 1,620
  9. 2018On First-Order Meta-Learning Algorithmscited by 1,018
  10. 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
  11. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 724
  12. 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 661
27 more
  1. 2018Meta-Learning with Latent Embedding Optimizationcited by 476
  2. 2019Meta-Learning with Differentiable Convex Optimizationcited by 461
  3. 2020End-to-End Video Instance Segmentation with Transformerscited by 443
  4. 2021Unifying Vision-and-Language Tasks via Text Generationcited by 412
  5. 2018TADAM: Task dependent adaptive metric for improved few-shot learningcited by 408
  6. 2018Few-Shot Learning via Embedding Adaptation with Set-to-Set Functionscited by 294
  7. 2019Cross Attention Network for Few-shot Classificationcited by 219
  8. 2020Meta-Baseline: Exploring Simple Meta-Learning for Few-Shot Learningcited by 179
  9. 2020CrossTransformers: spatially-aware few-shot transfercited by 174
  10. 2019Boosting Few-Shot Visual Learning with Self-Supervisioncited by 140
  11. 2021Visformer: The Vision-friendly Transformercited by 137
  12. 2019Adaptive Cross-Modal Few-Shot Learningcited by 95
  13. 2020Boosting Few-Shot Learning With Adaptive Margin Losscited by 83
  14. 2021KVT: k-NN Attention for Boosting Vision Transformerscited by 65
  15. 2019Associative Alignment for Few-shot Image Classificationcited by 58
  16. 2019Few-Shot Learning with Global Class Representationscited by 44
  17. 2023MetaDiff: Meta-Learning with Conditional Diffusion for Few-Shot Learningcited by 40
  18. 2022Rethinking Generalization in Few-Shot Classificationcited by 36
  19. 2019Difficulty-aware Meta-learning for Rare Disease Diagnosiscited by 24
  20. 2023Few Shot Part Segmentation Reveals Compositional Logic for Industrial Anomaly Detectioncited by 19
  21. 2021Few-Shot Learning with a Strong Teachercited by 16
  22. 2023ProtoDiff: Learning to Learn Prototypical Networks by Task-Guided Diffusioncited by 16
  23. 2021Aligning Visual Prototypes with BERT Embeddings for Few-Shot Learningcited by 12
  24. 2022RankDNN: Learning to Rank for Few-shot Learningcited by 7
  25. 2023ESPT: A Self-Supervised Episodic Spatial Pretext Task for Improving Few-Shot Learningcited by 6
  26. 2023SpatialFormer: Semantic and Target Aware Attentions for Few-Shot Learningcited by 4
  27. 2022Rethinking the Metric in Few-shot Learning: From an Adaptive Multi-Distance Perspectivecited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025VT-FSL: Bridging Vision and Text with LLMs for Few-Shot Learningcited by 7
  2. 2025Global Prompt Refinement with Non-Interfering Attention Masking for One-Shot Federated Learningcited by 2
  3. 2026DVLA-RL: Dual-Level Vision-Language Alignment with Reinforcement Learning Gating for Few-Shot Learningcited by 2