mlsanityData to 14 Oct 2026

Enhancing Few-shot Image Classification with Cosine Transformer

arXiv 2211.06828 ↗First posted 13 Nov 2022cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

This paper addresses the few-shot image classification problem, where the classification task is performed on unlabeled query samples given a small amount of labeled support samples only. One major challenge of the few-shot learning problem is the large variety of object visual appearances that prevents the support samples to represent that object comprehensively. This might result in a significant difference between support and query samples, therefore undermining the performance of few-shot algorithms. In this paper, we tackle the problem by proposing Few-shot Cosine Transformer (FS-CT), where the relational map between supports and queries is effectively obtained for the few-shot tasks. The FS-CT consists of two parts, a learnable prototypical embedding network to obtain categorical representations from support samples with hard cases, and a transformer encoder to effectively achieve the relational map from two different support and query samples. We introduce Cosine Attention, a more robust and stable attention module that enhances the transformer module significantly and therefore improves FS-CT performance from 5% to over 20% in accuracy compared to the default scaled dot-product mechanism. Our method performs competitive results in mini-ImageNet, CUB-200, and CIFAR-FS on 1-shot learning and 5-shot learning tasks across backbones and few-shot configurations. We also developed a custom few-shot dataset for Yoga pose recognition to demonstrate the potential of our algorithm for practical application. Our FS-CT with cosine attention is a lightweight, simple few-shot algorithm that can be applied for a wide range of applications, such as healthcare, medical, and security surveillance. The official implementation code of our Few-shot Cosine Transformer is available at https://github.com/vinuni-vishc/Few-Shot-Cosine-Transformer

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 43 of its 64 references that are in the map. First posted on arXiv 13 Nov 2022.

Builds on

43 of 64 references.

References · 43

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  6. 2020End-to-End Object Detection with Transformerscited by 7,414
  7. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  8. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  9. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  10. 2016Matching Networks for One Shot Learningcited by 2,255
  11. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
  12. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,143
31 more
  1. 2019A Closer Look at Few-shot Classificationcited by 684
  2. 2019Generalizing from a Few Examples: A Survey on Few-Shot Learningcited by 463
  3. 2020Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?cited by 437
  4. 2018Meta-learning with differentiable closed-form solverscited by 397
  5. 2018Dynamic Few-Shot Visual Learning without Forgettingcited by 385
  6. 2018Meta-Transfer Learning for Few-Shot Learningcited by 306
  7. 2018Few-Shot Learning via Embedding Adaptation with Set-to-Set Functionscited by 294
  8. 2018Probabilistic Model-Agnostic Meta-Learningcited by 256
  9. 2019Cross Attention Network for Few-shot Classificationcited by 219
  10. 2018Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learningcited by 211
  11. 2018Low-Shot Learning from Imaginary Datacited by 181
  12. 2020CrossTransformers: spatially-aware few-shot transfercited by 174
  13. 2021Few-Shot Segmentation via Cycle-Consistent Transformercited by 133
  14. 2022Pushing the Limits of Simple Pipelines for Few-Shot Learning: External Data and Fine-Tuning Make a Differencecited by 126
  15. 2017Few-Shot Image Recognition by Predicting Parameters from Activationscited by 122
  16. 2021Relational Embedding for Few-Shot Classificationcited by 74
  17. 2019Transductive Episodic-Wise Adaptive Metric for Few-Shot Learningcited by 67
  18. 2020A Universal Representation Transformer Layer for Few-Shot Image Classificationcited by 65
  19. 2022Few-Shot Object Detection with Fully Cross-Transformercited by 64
  20. 2021Recurrent Mask Refinement for Few-Shot Medical Image Segmentationcited by 63
  21. 2020Few-shot Scene-adaptive Anomaly Detectioncited by 62
  22. 2021A Hierarchical Transformation-Discriminating Generative Model for Few Shot Anomaly Detectioncited by 60
  23. 2017Cosine Normalization: Using Cosine Similarity Instead of Dot Product in Neural Networkscited by 59
  24. 2021Shallow Bayesian Meta Learning for Real-World Few-Shot Recognitioncited by 48
  25. 2020Enhancing Few-Shot Image Classification with Unlabelled Examplescited by 32
  26. 2021Few-shot Image Classification: Just Use a Library of Pre-trained Feature Extractors and a Simple Classifiercited by 32
  27. 2022Matching Feature Sets for Few-Shot Image Classificationcited by 25
  28. 2021Deep Metric Learning for Few-Shot Image Classification: A Review of Recent Developmentscited by 23
  29. 2021Subgraph-aware Few-Shot Inductive Link Prediction via Meta-Learningcited by 9
  30. 2021MHFC: Multi-Head Feature Collaboration for Few-Shot Learningcited by 6
  31. 2021Transductive Few-Shot Learning: Clustering is All You Need?cited by 4

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 1; 2025: 2.

4 papers.

  1. 2023Analyzing and Improving the Training Dynamics of Diffusion Modelscited by 402
  2. 2024Cottention: Linear Transformers With Cosine Attentioncited by 5
  3. 2025Local Foreground Selection aware Attentive Feature Reconstruction for few-shot fine-grained plant species classificationcited by 0
  4. 2025Exploring Magnitude Preservation and Rotation Modulation in Diffusion Transformerscited by 0