mlsanityData to 7 Oct 2026

A Survey on Transfer Learning in Natural Language Processing

arXiv 2007.04239 ↗First posted 31 May 2020cs.CL cs.LG stat.MLcited by 33

Not on the map. Why?

Abstract (arXiv)

Deep learning models usually require a huge amount of data. However, these large datasets are not always attainable. This is common in many challenging NLP tasks. Consider Neural Machine Translation, for instance, where curating such large datasets may not be possible specially for low resource languages. Another limitation of deep learning models is the demand for huge computing resources. These obstacles motivate research to question the possibility of knowledge transfer using large trained models. The demand for transfer learning is increasing as many large models are emerging. In this survey, we feature the recent transfer learning advances in the field of NLP. We also provide a taxonomy for categorizing different transfer learning approaches from the literature.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). First posted on arXiv 31 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 5; 2023: 11; 2024: 6; 2025: 7; 2026: 1.

33 papers.

Sort:

  1. 2022Head2Toe: Utilizing Intermediate Representations for Better Transfer Learningcited by 83
  2. 2023Deep Transfer Learning for Automatic Speech Recognition: Towards Better Generalizationcited by 28
  3. 2023Position: Key Claims in LLM Research Have a Long Tail of Footnotescited by 20
  4. 2022Hate Speech and Offensive Language Detection in Bengalicited by 19
  5. 2021Privacy enabled Financial Text Classification using Differential Privacy and Federated Learningcited by 18
  6. 2023Towards energy-efficient Deep Learning: An overview of energy-efficient approaches along the Deep Learning Lifecyclecited by 13
  7. 2023Guiding The Last Layer in Federated Learning with Pre-Trained Modelscited by 13
  8. 2023Neural Machine Translation of Clinical Text: An Empirical Investigation into Multilingual Pre-Trained Language Models and Transfer-Learningcited by 8
  9. 2024NLP for The Greek Language: A Longer Surveycited by 8
  10. 2023A Recent Survey of Heterogeneous Transfer Learningcited by 7
  11. 2024SPARQL Generation with Entity Pre-trained GPT for KG Question Answeringcited by 6
  12. 2025Unlocking the Potential of Generative AI through Neuro-Symbolic Architectures: Benefits and Limitationscited by 6
21 more
  1. 2022Transfer Learning for Individual Treatment Effect Estimationcited by 5
  2. 2024Exploring Tokenization Strategies and Vocabulary Sizes for Enhanced Arabic Language Modelscited by 5
  3. 2023Towards the Fundamental Limits of Knowledge Transfer over Finite Domainscited by 4
  4. 2023How to Determine the Most Powerful Pre-trained Language Model without Brute Force Fine-tuning? An Empirical Surveycited by 4
  5. 2025Foundation Models for Environmental Science: A Survey of Emerging Frontierscited by 4
  6. 2025Neuroplasticity in Artificial Intelligence -- An Overview and Inspirations on Drop In & Out Learningcited by 3
  7. 2021Literature review on vulnerability detection using NLP technologycited by 2
  8. 2021Deception detection in text and its relation to the cultural dimension of individualism/collectivismcited by 2
  9. 2023Multimodal Prompt Learning for Product Title Generation with Extremely Limited Labelscited by 2
  10. 2023An Introduction to Natural Language Processing Techniques and Framework for Clinical Implementation in Radiation Oncologycited by 2
  11. 2024FAQ-Gen: An automated system to generate domain-specific FAQs to aid content comprehensioncited by 2
  12. 2024Transfer Learning with Clinical Concept Embeddings from Large Language Modelscited by 2
  13. 2022Punctuation Restoration in Spanish Customer Support Transcripts using Transfer Learningcited by 1
  14. 2024Online Transfer Learning for RSV Case Detectioncited by 1
  15. 2025LEKA:LLM-Enhanced Knowledge Augmentationcited by 1
  16. 2022Crude Oil-related Events Extraction and Processing: A Transfer Learning Approachcited by 0
  17. 2023Natural Language Processing Through Transfer Learning: A Case Study on Sentiment Analysiscited by 0
  18. 2025Poem Meter Classification of Recited Arabic Poetry: Integrating High-Resource Systems for a Low-Resource Taskcited by 0
  19. 2025Speaker Diarization for Low-Resource Languages Through Wav2vec Fine-Tuningcited by 0
  20. 2025LMSpell: Spell Correction with Pre-Trained Language Modelscited by 0
  21. 2026RADS: Reinforcement Learning-Based Sample Selection Improves Transfer Learning in Low-resource and Imbalanced Clinical Settingscited by 0