# When & How to Transfer with Transfer Learning

arXiv: https://arxiv.org/abs/2211.04347 · HTML: https://mlsanity.com/paper/2211.04347/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 34 references that are in the map.
- First posted on arXiv 8 Nov 2022.

## Builds on (most cited first, 19 of 19)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [Revisiting Unreasonable Effectiveness of Data in Deep Learning Era](https://mlsanity.com/paper/1707.02968/) (2017)
- [Exploring the Limits of Weakly Supervised Pretraining](https://mlsanity.com/paper/1805.00932/) (2018)
- [DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition](https://mlsanity.com/paper/1310.1531/) (2013)
- [What is being transferred in transfer learning?](https://mlsanity.com/paper/2008.11687/) (2020)
- [CNN Features off-the-shelf: an Astounding Baseline for Recognition](https://mlsanity.com/paper/1403.6382/) (2014)
- [Large Scale Fine-Grained Categorization and Domain-Specific Transfer Learning](https://mlsanity.com/paper/1806.06193/) (2018)
- [Places: An Image Database for Deep Scene Understanding](https://mlsanity.com/paper/1610.02055/) (2016)
- [The Unreasonable Effectiveness of Noisy Data for Fine-Grained Recognition](https://mlsanity.com/paper/1511.06789/) (2015)
- [FOSNet: An End-to-End Trainable Deep Neural Network for Scene Recognition](https://mlsanity.com/paper/1907.07570/) (2019)
- [Part-Aware Fine-grained Object Categorization using Weakly Supervised Part Detection Network](https://mlsanity.com/paper/1806.06198/) (2018)
- [The MAMe Dataset: On the relevance of High Resolution and Variable Shape image properties](https://mlsanity.com/paper/2007.13693/) (2020)
- [An Out-of-the-box Full-network Embedding for Convolutional Neural Networks](https://mlsanity.com/paper/1705.07706/) (2017)
- [Domain Adaptive Transfer Learning on Visual Attention Aware Data Augmentation for Fine-grained Visual Categorization](https://mlsanity.com/paper/2010.03071/) (2020)

## Abstract (arXiv)

In deep learning, transfer learning (TL) has become the de facto approach when dealing with image related tasks. Visual features learnt for one task have been shown to be reusable for other tasks, improving performance significantly. By reusing deep representations, TL enables the use of deep models in domains with limited data availability, limited computational resources and/or limited access to human experts. Domains which include the vast majority of real-life applications. This paper conducts an experimental evaluation of TL, exploring its trade-offs with respect to performance, environmental footprint, human hours and computational requirements. Results highlight the cases were a cheap feature extraction approach is preferable, and the situations where an expensive fine-tuning effort may be worth the added cost. Finally, a set of guidelines on the use of TL are proposed.