mlsanityData to 7 Oct 2026

CAD: Co-Adapting Discriminative Features for Improved Few-Shot Classification

arXiv 2203.13465 ↗First posted 25 Mar 2022cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

Few-shot classification is a challenging problem that aims to learn a model that can adapt to unseen classes given a few labeled samples. Recent approaches pre-train a feature extractor, and then fine-tune for episodic meta-learning. Other methods leverage spatial features to learn pixel-level correspondence while jointly training a classifier. However, results using such approaches show marginal improvements. In this paper, inspired by the transformer style self-attention mechanism, we propose a strategy to cross-attend and re-weight discriminative features for few-shot classification. Given a base representation of support and query images after global pooling, we introduce a single shared module that projects features and cross-attends in two aspects: (i) query to support, and (ii) support to query. The module computes attention scores between features to produce an attention pooled representation of features in the same class that is later added to the original representation followed by a projection head. This effectively re-weights features in both aspects (i & ii) to produce features that better facilitate improved metric-based meta-learning. Extensive experiments on public benchmarks show our approach outperforms state-of-the-art methods by 3%~5%.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 48 of its 58 references that are in the map. First posted on arXiv 25 Mar 2022.

Builds on

48 of 58 references.

References · 48

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  5. 2017Non-local Neural Networkscited by 2,290
  6. 2016Matching Networks for One Shot Learningcited by 2,246
  7. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,139
  8. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  9. 2019A Closer Look at Few-shot Classificationcited by 679
  10. 2019Meta-Learning with Differentiable Convex Optimizationcited by 461
  11. 2018Meta-Learning for Semi-Supervised Few-Shot Classificationcited by 456
  12. 2020Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?cited by 435
36 more
  1. 2018Meta-learning with differentiable closed-form solverscited by 395
  2. 2018Dynamic Few-Shot Visual Learning without Forgettingcited by 385
  3. 2019Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAMLcited by 359
  4. 2019Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examplescited by 312
  5. 2019A Baseline for Few-Shot Image Classificationcited by 298
  6. 2018Few-Shot Learning via Embedding Adaptation with Set-to-Set Functionscited by 293
  7. 2018Probabilistic Model-Agnostic Meta-Learningcited by 254
  8. 2019Cross Attention Network for Few-shot Classificationcited by 218
  9. 2018Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learningcited by 208
  10. 2019SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learningcited by 194
  11. 2020CrossTransformers: spatially-aware few-shot transfercited by 174
  12. 2021Free Lunch for Few-shot Learning: Distribution Calibrationcited by 165
  13. 2019Boosting Few-Shot Visual Learning with Self-Supervisioncited by 140
  14. 2019Charting the Right Manifold: Manifold Mixup for Few-shot Learningcited by 135
  15. 2018Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networkscited by 129
  16. 2020Negative Margin Matters: Understanding Margin in Few-shot Classificationcited by 129
  17. 2020Few-Shot Classification with Feature Map Reconstruction Networkscited by 116
  18. 2019Improved Few-Shot Visual Classificationcited by 114
  19. 2020Adversarial Feature Hallucination Networks for Few-Shot Learningcited by 110
  20. 2019Finding Task-Relevant Features for Few-Shot Learning by Category Traversalcited by 101
  21. 2018Delta-encoder: an effective sample synthesis method for few-shot object recognitioncited by 89
  22. 2020Embedding Propagation: Smoother Manifold for Few-Shot Classificationcited by 79
  23. 2021Relational Embedding for Few-Shot Classificationcited by 74
  24. 2021Partial Is Better Than All: Revisiting Fine-tuning Strategy for Few-shot Learningcited by 72
  25. 2018Incremental Few-Shot Learning with Attention Attractor Networkscited by 65
  26. 2020A Universal Representation Transformer Layer for Few-Shot Image Classificationcited by 65
  27. 2019Associative Alignment for Few-shot Image Classificationcited by 58
  28. 2019Few-Shot Learning with Global Class Representationscited by 44
  29. 2019Self-Supervised Learning For Few-Shot Image Classificationcited by 44
  30. 2018Few-Shot Learning with Metric-Agnostic Conditional Embeddingscited by 40
  31. 2020Enhancing Few-Shot Image Classification with Unlabelled Examplescited by 32
  32. 2019Towards Robust Image Classification Using Sequential Attention Modelscited by 27
  33. 2021Reinforced Attention for Few-Shot Learning and Beyondcited by 18
  34. 2020Bidirectional RNN-based Few Shot Learning for 3D Medical Image Segmentationcited by 12
  35. 2020Few-Shot Relation Learning with Attention for EEG-based Motor Imagery Classificationcited by 10
  36. 2021Transductive Few-Shot Learning: Clustering is All You Need?cited by 4

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 4; 2024: 2.

9 papers.

  1. 2022A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunitiescited by 162
  2. 2024FedMM: Federated Multi-Modal Learning with Modality Heterogeneity in Computational Pathologycited by 17
  3. 2022Few-Shot Learning Meets Transformer: Unified Query-Support Transformers for Few-Shot Classificationcited by 16
  4. 2023DETA: Denoised Task Adaptation for Few-Shot Learningcited by 13
  5. 2023Hubs and Hyperspheres: Reducing Hubness and Improving Transductive Few-shot Learning with Hyperspherical Embeddingscited by 11
  6. 2022Few-shot Metric Learning: Online Adaptation of Embedding for Retrievalcited by 8
  7. 2024Knowledge Adaptation Network for Few-Shot Class-Incremental Learningcited by 1
  8. 2023Recent Advances of Local Mechanisms in Computer Vision: A Survey and Outlook of Recent Workcited by 0
  9. 2023Knowledge Transfer-Driven Few-Shot Class-Incremental Learningcited by 0