# How Transferable are Neural Networks in NLP Applications?

arXiv: https://arxiv.org/abs/1603.06111 · HTML: https://mlsanity.com/paper/1603.06111/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Mar 2016.

## Built on by (most cited first, 23 of 23)

- [Improving CLIP Training with Language Rewrites](https://mlsanity.com/paper/2305.20088/) (2023)
- [On the Effectiveness of Adapter-based Tuning for Pretrained Language Model Adaptation](https://mlsanity.com/paper/2106.03164/) (2021)
- [A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learning](https://mlsanity.com/paper/2408.07057/) (2024)
- [Neural Natural Language Processing for Unstructured Data in Electronic Health Records: a Review](https://mlsanity.com/paper/2107.02975/) (2021)
- [Cross-Domain Label-Adaptive Stance Detection](https://mlsanity.com/paper/2104.07467/) (2021)
- [Cross-Domain Multi-Task Learning for Sequential Sentence Classification in Research Papers](https://mlsanity.com/paper/2102.06008/) (2021)
- [Studying Catastrophic Forgetting in Neural Ranking Models](https://mlsanity.com/paper/2101.06984/) (2021)
- [On the Effect of Pre-training for Transformer in Different Modality on Offline Reinforcement Learning](https://mlsanity.com/paper/2211.09817/) (2022)
- [indicnlp@kgp at DravidianLangTech-EACL2021: Offensive Language Identification in Dravidian Languages](https://mlsanity.com/paper/2102.07150/) (2021)
- [Glider: Global and Local Instruction-Driven Expert Router](https://mlsanity.com/paper/2410.07172/) (2024)
- [Analysis and Prediction of NLP Models Via Task Embeddings](https://mlsanity.com/paper/2112.05647/) (2021)
- [Neural Supervised Domain Adaptation by Augmenting Pre-trained Models with Random Units](https://mlsanity.com/paper/2106.04935/) (2021)
- [Distributionally Robust Domain Adaptation](https://mlsanity.com/paper/2210.16894/) (2022)
- [Predicting the Success of Domain Adaptation in Text Similarity](https://mlsanity.com/paper/2106.04641/) (2021)
- [Fine-tuning Strategies for Domain Specific Question Answering under Low Annotation Budget Constraints](https://mlsanity.com/paper/2401.09168/) (2024)
- [Artificial Perceptual Learning: Image Categorization with Weak Supervision](https://mlsanity.com/paper/2106.07559/) (2021)
- [Enhanced Seq2Seq Autoencoder via Contrastive Learning for Abstractive Text Summarization](https://mlsanity.com/paper/2108.11992/) (2021)
- [Finding Materialized Models for Model Reuse](https://mlsanity.com/paper/2110.06532/) (2021)
- [Identifying Suitable Tasks for Inductive Transfer Through the Analysis of Feature Attributions](https://mlsanity.com/paper/2202.01096/) (2022)
- [Non-parametric, Nearest-neighbor-assisted Fine-tuning for Neural Machine Translation](https://mlsanity.com/paper/2305.13648/) (2023)
- [Cross-Demographic Portability of Deep NLP-Based Depression Models](https://mlsanity.com/paper/2412.19070/) (2024)
- [Robust Speech and Natural Language Processing Models for Depression Screening](https://mlsanity.com/paper/2412.19072/) (2024)
- [Rethinking Reasoning in LLMs: Neuro-Symbolic Local RetoMaton Beyond ICL and CoT](https://mlsanity.com/paper/2508.19271/) (2025)

## Abstract (arXiv)

Transfer learning is aimed to make use of valuable knowledge in a source domain to help model performance in a target domain. It is particularly important to neural networks, which are very likely to be overfitting. In some fields like image processing, many studies have shown the effectiveness of neural network-based transfer learning. For neural NLP, however, existing studies have only casually applied transfer learning, and conclusions are inconsistent. In this paper, we conduct systematic case studies and provide an illuminating picture on the transferability of neural networks in NLP.