# A Practitioners' Guide to Transfer Learning for Text Classification using Convolutional Neural Networks

arXiv: https://arxiv.org/abs/1801.06480 · HTML: https://mlsanity.com/paper/1801.06480/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Jan 2018.

## Built on by (most cited first, 7 of 7)

- [Cross-Domain Multi-Task Learning for Sequential Sentence Classification in Research Papers](https://mlsanity.com/paper/2102.06008/) (2021)
- [Identifying Self-Admitted Technical Debt in Issue Tracking Systems using Machine Learning](https://mlsanity.com/paper/2202.02180/) (2022)
- [Reset It and Forget It: Relearning Last-Layer Weights Improves Continual and Transfer Learning](https://mlsanity.com/paper/2310.07996/) (2023)
- [A Bayesian approach for prompt optimization in pre-trained language models](https://mlsanity.com/paper/2312.00471/) (2023)
- [Sentiment Analysis of Movie Reviews Using BERT](https://mlsanity.com/paper/2502.18841/) (2025)
- [A Novel Deep Learning Method for Textual Sentiment Analysis](https://mlsanity.com/paper/2102.11651/) (2021)
- [Fine-tuning BERT with Bidirectional LSTM for Fine-grained Movie Reviews Sentiment Analysis](https://mlsanity.com/paper/2502.20682/) (2025)

## Abstract (arXiv)

Transfer Learning (TL) plays a crucial role when a given dataset has insufficient labeled examples to train an accurate model. In such scenarios, the knowledge accumulated within a model pre-trained on a source dataset can be transferred to a target dataset, resulting in the improvement of the target model. Though TL is found to be successful in the realm of image-based applications, its impact and practical use in Natural Language Processing (NLP) applications is still a subject of research. Due to their hierarchical architecture, Deep Neural Networks (DNN) provide flexibility and customization in adjusting their parameters and depth of layers, thereby forming an apt area for exploiting the use of TL. In this paper, we report the results and conclusions obtained from extensive empirical experiments using a Convolutional Neural Network (CNN) and try to uncover thumb rules to ensure a successful positive transfer. In addition, we also highlight the flawed means that could lead to a negative transfer. We explore the transferability of various layers and describe the effect of varying hyper-parameters on the transfer performance. Also, we present a comparison of accuracy value and model size against state-of-the-art methods. Finally, we derive inferences from the empirical results and provide best practices to achieve a successful positive transfer.