# CAD: Co-Adapting Discriminative Features for Improved Few-Shot Classification

arXiv: https://arxiv.org/abs/2203.13465 · HTML: https://mlsanity.com/paper/2203.13465/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 58 references that are in the map.
- First posted on arXiv 25 Mar 2022.

## Builds on (most cited first, 30 of 48)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [A Closer Look at Few-shot Classification](https://mlsanity.com/paper/1904.04232/) (2019)
- [Meta-Learning with Differentiable Convex Optimization](https://mlsanity.com/paper/1904.03758/) (2019)
- [Meta-Learning for Semi-Supervised Few-Shot Classification](https://mlsanity.com/paper/1803.00676/) (2018)
- [Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?](https://mlsanity.com/paper/2003.11539/) (2020)
- [Meta-learning with differentiable closed-form solvers](https://mlsanity.com/paper/1805.08136/) (2018)
- [Dynamic Few-Shot Visual Learning without Forgetting](https://mlsanity.com/paper/1804.09458/) (2018)
- [Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML](https://mlsanity.com/paper/1909.09157/) (2019)
- [Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples](https://mlsanity.com/paper/1903.03096/) (2019)
- [A Baseline for Few-Shot Image Classification](https://mlsanity.com/paper/1909.02729/) (2019)
- [Few-Shot Learning via Embedding Adaptation with Set-to-Set Functions](https://mlsanity.com/paper/1812.03664/) (2018)
- [Probabilistic Model-Agnostic Meta-Learning](https://mlsanity.com/paper/1806.02817/) (2018)
- [Cross Attention Network for Few-shot Classification](https://mlsanity.com/paper/1910.07677/) (2019)
- [Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learning](https://mlsanity.com/paper/1805.10002/) (2018)
- [SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learning](https://mlsanity.com/paper/1911.04623/) (2019)
- [CrossTransformers: spatially-aware few-shot transfer](https://mlsanity.com/paper/2007.11498/) (2020)
- [Free Lunch for Few-shot Learning: Distribution Calibration](https://mlsanity.com/paper/2101.06395/) (2021)
- [Boosting Few-Shot Visual Learning with Self-Supervision](https://mlsanity.com/paper/1906.05186/) (2019)
- [Charting the Right Manifold: Manifold Mixup for Few-shot Learning](https://mlsanity.com/paper/1907.12087/) (2019)
- [Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks](https://mlsanity.com/paper/1810.00825/) (2018)
- [Negative Margin Matters: Understanding Margin in Few-shot Classification](https://mlsanity.com/paper/2003.12060/) (2020)
- [Few-Shot Classification with Feature Map Reconstruction Networks](https://mlsanity.com/paper/2012.01506/) (2020)
- [Improved Few-Shot Visual Classification](https://mlsanity.com/paper/1912.03432/) (2019)

## Built on by (most cited first, 9 of 9)

- [A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities](https://mlsanity.com/paper/2205.06743/) (2022)
- [FedMM: Federated Multi-Modal Learning with Modality Heterogeneity in Computational Pathology](https://mlsanity.com/paper/2402.15858/) (2024)
- [Few-Shot Learning Meets Transformer: Unified Query-Support Transformers for Few-Shot Classification](https://mlsanity.com/paper/2208.12398/) (2022)
- [DETA: Denoised Task Adaptation for Few-Shot Learning](https://mlsanity.com/paper/2303.06315/) (2023)
- [Hubs and Hyperspheres: Reducing Hubness and Improving Transductive Few-shot Learning with Hyperspherical Embeddings](https://mlsanity.com/paper/2303.09352/) (2023)
- [Few-shot Metric Learning: Online Adaptation of Embedding for Retrieval](https://mlsanity.com/paper/2211.07116/) (2022)
- [Knowledge Adaptation Network for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2409.11770/) (2024)
- [Recent Advances of Local Mechanisms in Computer Vision: A Survey and Outlook of Recent Work](https://mlsanity.com/paper/2306.01929/) (2023)
- [Knowledge Transfer-Driven Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2306.10942/) (2023)

## Abstract (arXiv)

Few-shot classification is a challenging problem that aims to learn a model that can adapt to unseen classes given a few labeled samples. Recent approaches pre-train a feature extractor, and then fine-tune for episodic meta-learning. Other methods leverage spatial features to learn pixel-level correspondence while jointly training a classifier. However, results using such approaches show marginal improvements. In this paper, inspired by the transformer style self-attention mechanism, we propose a strategy to cross-attend and re-weight discriminative features for few-shot classification. Given a base representation of support and query images after global pooling, we introduce a single shared module that projects features and cross-attends in two aspects: (i) query to support, and (ii) support to query. The module computes attention scores between features to produce an attention pooled representation of features in the same class that is later added to the original representation followed by a projection head. This effectively re-weights features in both aspects (i & ii) to produce features that better facilitate improved metric-based meta-learning. Extensive experiments on public benchmarks show our approach outperforms state-of-the-art methods by 3%~5%.