# Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?

arXiv: https://arxiv.org/abs/2003.11539 · HTML: https://mlsanity.com/paper/2003.11539/

## Key facts

- Cited by 437 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Mar 2020.

## Built on by (most cited first, 30 of 437)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [Intriguing Properties of Vision Transformers](https://mlsanity.com/paper/2105.10497/) (2021)
- [Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling](https://mlsanity.com/paper/2111.03930/) (2021)
- [The Platonic Representation Hypothesis](https://mlsanity.com/paper/2405.07987/) (2024)
- [Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classification](https://mlsanity.com/paper/2207.09519/) (2022)
- [Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classification](https://mlsanity.com/paper/2211.11158/) (2022)
- [MultiMAE: Multi-modal Multi-task Masked Autoencoders](https://mlsanity.com/paper/2204.01678/) (2022)
- [Prompt Distribution Learning](https://mlsanity.com/paper/2205.03340/) (2022)
- [CrossPoint: Self-Supervised Cross-Modal Contrastive Learning for 3D Point Cloud Understanding](https://mlsanity.com/paper/2203.00680/) (2022)
- [FedBABU: Towards Enhanced Representation for Federated Image Classification](https://mlsanity.com/paper/2106.06042/) (2021)
- [StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learners](https://mlsanity.com/paper/2306.00984/) (2023)
- [A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities](https://mlsanity.com/paper/2205.06743/) (2022)
- [Pushing the Limits of Simple Pipelines for Few-Shot Learning: External Data and Fine-Tuning Make a Difference](https://mlsanity.com/paper/2204.07305/) (2022)
- [4M: Massively Multimodal Masked Modeling](https://mlsanity.com/paper/2312.06647/) (2023)
- [Using Shape to Categorize: Low-Shot Learning with an Explicit Shape Bias](https://mlsanity.com/paper/2101.07296/) (2021)
- [Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minima](https://mlsanity.com/paper/2111.01549/) (2021)
- [Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformer](https://mlsanity.com/paper/2108.03032/) (2021)
- [PartImageNet: A Large, High-Quality Dataset of Parts](https://mlsanity.com/paper/2112.00933/) (2021)
- [DeFRCN: Decoupled Faster R-CNN for Few-Shot Object Detection](https://mlsanity.com/paper/2108.09017/) (2021)
- [On the Role of Neural Collapse in Transfer Learning](https://mlsanity.com/paper/2112.15121/) (2021)
- [Constrained Few-shot Class-incremental Learning](https://mlsanity.com/paper/2203.16588/) (2022)
- [Learning from Few Examples: A Summary of Approaches to Few-Shot Learning](https://mlsanity.com/paper/2203.04291/) (2022)
- [Cross-domain Few-shot Learning with Task-specific Adapters](https://mlsanity.com/paper/2107.00358/) (2021)
- [A Broad Study on the Transferability of Visual Representations with Contrastive Learning](https://mlsanity.com/paper/2103.13517/) (2021)
- [Elaborative Rehearsal for Zero-shot Action Recognition](https://mlsanity.com/paper/2108.02833/) (2021)
- [Relational Embedding for Few-Shot Classification](https://mlsanity.com/paper/2108.09666/) (2021)
- [Partial Is Better Than All: Revisiting Fine-tuning Strategy for Few-shot Learning](https://mlsanity.com/paper/2102.03983/) (2021)
- [Can Foundation Models Perform Zero-Shot Task Specification For Robot Manipulation?](https://mlsanity.com/paper/2204.11134/) (2022)

## Abstract (arXiv)

The focus of recent meta-learning research has been on the development of learning algorithms that can quickly adapt to test time tasks with limited data and low computational cost. Few-shot learning is widely used as one of the standard benchmarks in meta-learning. In this work, we show that a simple baseline: learning a supervised or self-supervised representation on the meta-training set, followed by training a linear classifier on top of this representation, outperforms state-of-the-art few-shot learning methods. An additional boost can be achieved through the use of self-distillation. This demonstrates that using a good learned embedding model can be more effective than sophisticated meta-learning algorithms. We believe that our findings motivate a rethinking of few-shot image classification benchmarks and the associated role of meta-learning algorithms. Code is available at: http://github.com/WangYueFt/rfs/.