# Few-shot Image Classification: Just Use a Library of Pre-trained Feature Extractors and a Simple Classifier

arXiv: https://arxiv.org/abs/2101.00562 · HTML: https://mlsanity.com/paper/2101.00562/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 40 references that are in the map.
- First posted on arXiv 3 Jan 2021.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [Revisiting Unreasonable Effectiveness of Data in Deep Learning Era](https://mlsanity.com/paper/1707.02968/) (2017)
- [Big Transfer (BiT): General Visual Representation Learning](https://mlsanity.com/paper/1912.11370/) (2019)
- [A Closer Look at Few-shot Classification](https://mlsanity.com/paper/1904.04232/) (2019)
- [Meta-Learning with Latent Embedding Optimization](https://mlsanity.com/paper/1807.05960/) (2018)
- [Meta-Learning with Differentiable Convex Optimization](https://mlsanity.com/paper/1904.03758/) (2019)
- [Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?](https://mlsanity.com/paper/2003.11539/) (2020)
- [TADAM: Task dependent adaptive metric for improved few-shot learning](https://mlsanity.com/paper/1805.10123/) (2018)
- [Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples](https://mlsanity.com/paper/1903.03096/) (2019)
- [Meta-Transfer Learning for Few-Shot Learning](https://mlsanity.com/paper/1812.02391/) (2018)
- [A Baseline for Few-Shot Image Classification](https://mlsanity.com/paper/1909.02729/) (2019)
- [Few-Shot Learning via Embedding Adaptation with Set-to-Set Functions](https://mlsanity.com/paper/1812.03664/) (2018)
- [Probabilistic Model-Agnostic Meta-Learning](https://mlsanity.com/paper/1806.02817/) (2018)
- [Bayesian Model-Agnostic Meta-Learning](https://mlsanity.com/paper/1806.03836/) (2018)
- [Low-Shot Learning from Imaginary Data](https://mlsanity.com/paper/1801.05401/) (2018)
- [Finding Task-Relevant Features for Few-Shot Learning by Category Traversal](https://mlsanity.com/paper/1905.11116/) (2019)
- [Diversity with Cooperation: Ensemble Methods for Few-Shot Classification](https://mlsanity.com/paper/1903.11341/) (2019)

## Built on by (most cited first, 30 of 32)

- [Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classification](https://mlsanity.com/paper/2211.11158/) (2022)
- [Compound Prototype Matching for Few-shot Action Recognition](https://mlsanity.com/paper/2207.05515/) (2022)
- [Memorizing Complementation Network for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2208.05610/) (2022)
- [Few-shot Relational Reasoning via Connection Subgraph Pretraining](https://mlsanity.com/paper/2210.06722/) (2022)
- [Learning useful representations for shifting tasks and distributions](https://mlsanity.com/paper/2212.07346/) (2022)
- [On the Convergence of Shallow Neural Network Training with Randomly Masked Neurons](https://mlsanity.com/paper/2112.02668/) (2021)
- [The Art of Prompting: Event Detection based on Type Specific Prompts](https://mlsanity.com/paper/2204.07241/) (2022)
- [SARF: Aliasing Relation Assisted Self-Supervised Learning for Few-shot Relation Reasoning](https://mlsanity.com/paper/2304.10297/) (2023)
- [Few-shot Metric Learning: Online Adaptation of Embedding for Retrieval](https://mlsanity.com/paper/2211.07116/) (2022)
- [Lifelong Ensemble Learning based on Multiple Representations for Few-Shot Object Recognition](https://mlsanity.com/paper/2205.01982/) (2022)
- [Enhancing Few-shot Image Classification with Cosine Transformer](https://mlsanity.com/paper/2211.06828/) (2022)
- [Federated Learning Over Images: Vertical Decompositions and Pre-Trained Backbones Are Difficult to Beat](https://mlsanity.com/paper/2309.03237/) (2023)
- [Enhanced Few-Shot Class-Incremental Learning via Ensemble Models](https://mlsanity.com/paper/2401.07208/) (2024)
- [Targeted Augmentation for Low-Resource Event Extraction](https://mlsanity.com/paper/2405.08729/) (2024)
- [Real-Time Audio-Visual Speech Enhancement Using Pre-trained Visual Representations](https://mlsanity.com/paper/2507.21448/) (2025)
- [Leveraging Normalization Layer in Adapters With Progressive Learning and Adaptive Distillation for Cross-Domain Few-Shot Learning](https://mlsanity.com/paper/2312.11260/) (2023)
- [Fine-tuning with Very Large Dropout](https://mlsanity.com/paper/2403.00946/) (2024)
- [Fantastic Features and Where to Find Them: A Probing Method to combine Features from Multiple Foundation Models](https://mlsanity.com/paper/2512.01405/) (2025)
- [Attentional Meta-learners for Few-shot Polythetic Classification](https://mlsanity.com/paper/2106.05317/) (2021)
- [Constructing Sample-to-Class Graph for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2310.20268/) (2023)
- [Convolutional Ensembling based Few-Shot Defect Detection Technique](https://mlsanity.com/paper/2208.03288/) (2022)
- [Advancing Image Retrieval with Few-Shot Learning and Relevance Feedback](https://mlsanity.com/paper/2312.11078/) (2023)
- [Music auto-tagging in the long tail: A few-shot approach](https://mlsanity.com/paper/2409.07730/) (2024)
- [Low-Data Classification of Historical Music Manuscripts: A Few-Shot Learning Approach](https://mlsanity.com/paper/2411.16408/) (2024)
- [Few-Shot Learning for Image Classification of Common Flora](https://mlsanity.com/paper/2105.03056/) (2021)
- [One Representation to Rule Them All: Identifying Out-of-Support Examples in Few-shot Learning with Generic Representations](https://mlsanity.com/paper/2106.01423/) (2021)
- [Improving traffic sign recognition by active search](https://mlsanity.com/paper/2111.14426/) (2021)
- [Feature Transformation for Cross-domain Few-shot Remote Sensing Scene Classification](https://mlsanity.com/paper/2203.02270/) (2022)
- [ELMF4EggQ: Ensemble Learning with Multimodal Feature Fusion for Non-Destructive Egg Quality Assessment](https://mlsanity.com/paper/2510.02876/) (2025)
- [A Domain-Adapted Lightweight Ensemble for Resource-Efficient Few-Shot Plant Disease Classification](https://mlsanity.com/paper/2512.13428/) (2025)

## Abstract (arXiv)

Recent papers have suggested that transfer learning can outperform sophisticated meta-learning methods for few-shot image classification. We take this hypothesis to its logical conclusion, and suggest the use of an ensemble of high-quality, pre-trained feature extractors for few-shot image classification. We show experimentally that a library of pre-trained feature extractors combined with a simple feed-forward network learned with an L2-regularizer can be an excellent option for solving cross-domain few-shot image classification. Our experimental results suggest that this simpler sample-efficient approach far outperforms several well-established meta-learning algorithms on a variety of few-shot tasks.