# An empirical study of pretrained representations for few-shot classification

arXiv: https://arxiv.org/abs/1910.01319 · HTML: https://mlsanity.com/paper/1910.01319/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Oct 2019.

## Built on by (most cited first, 3 of 3)

- [Zoo-Tuning: Adaptive Transfer from a Zoo of Models](https://mlsanity.com/paper/2106.15434/) (2021)
- [Hub-Pathway: Transfer Learning from A Hub of Pre-trained Models](https://mlsanity.com/paper/2206.03726/) (2022)
- [A Deep Learning Framework for Lifelong Machine Learning](https://mlsanity.com/paper/2105.00157/) (2021)

## Abstract (arXiv)

Recent algorithms with state-of-the-art few-shot classification results start their procedure by computing data features output by a large pretrained model. In this paper we systematically investigate which models provide the best representations for a few-shot image classification task when pretrained on the Imagenet dataset. We test their representations when used as the starting point for different few-shot classification algorithms. We observe that models trained on a supervised classification task have higher performance than models trained in an unsupervised manner even when transferred to out-of-distribution datasets. Models trained with adversarial robustness transfer better, while having slightly lower accuracy than supervised models.