mlsanityData to 7 Oct 2026

A Review of Deep Transfer Learning and Recent Advancements

arXiv 2201.09679 ↗First posted 19 Jan 2022cs.LG cs.AI cs.CVcited by 56

Not on the map. Why?

Abstract (arXiv)

Deep learning has been the answer to many machine learning problems during the past two decades. However, it comes with two major constraints: dependency on extensive labeled data and training costs. Transfer learning in deep learning, known as Deep Transfer Learning (DTL), attempts to reduce such dependency and costs by reusing an obtained knowledge from a source data/task in training on a target data/task. Most applied DTL techniques are network/model-based approaches. These methods reduce the dependency of deep learning models on extensive training data and drastically decrease training costs. As a result, researchers detected Covid-19 infection on chest X-Rays with high accuracy at the beginning of the pandemic with minimal data using DTL techniques. Also, the training cost reduction makes DTL viable on edge devices with limited resources. Like any new advancement, DTL methods have their own limitations, and a successful transfer depends on some adjustments for different scenarios. In this paper, we review the definition and taxonomy of deep transfer learning and well-known methods. Then we investigate the DTL approaches by reviewing recent applied DTL techniques in the past five years. Further, we review some experimental analyses of DTLs to learn the best practice for applying DTL in different scenarios. Moreover, the limitations of DTLs (catastrophic forgetting dilemma and overly biased pre-trained models) are discussed, along with possible solutions and research trends.

Key facts. Cited by 56 papers in the map (data through 7 Oct 2026). Builds on 24 of its 72 references that are in the map. First posted on arXiv 19 Jan 2022.

Builds on

24 of 72 references.

References · 24

  1. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  2. 2016Progressive Neural Networkscited by 1,332
  3. 2014How transferable are features in deep neural networks?cited by 1,216
  4. 2019A Comprehensive Survey on Transfer Learningcited by 1,154
  5. 2016An overview of gradient descent optimization algorithmscited by 1,021
  6. 2018A Survey on Deep Transfer Learningcited by 417
  7. 2020What is being transferred in transfer learning?cited by 365
  8. 2015Understanding Neural Networks Through Deep Visualizationcited by 332
  9. 2020A Brief Review of Domain Adaptationcited by 287
  10. 2019Learning to Discover Novel Visual Categories via Deep Transfer Clusteringcited by 223
  11. 2019Toward Understanding Catastrophic Forgetting in Continual Learningcited by 67
  12. 2021Factors of Influence for Transfer Learning across Diverse Appearance Domains and Task Typescited by 54
12 more
  1. 2019Growing a Brain: Fine-Tuning by Increasing Model Capacitycited by 43
  2. 2019Deep Transfer Learning for Multiple Class Novelty Detectioncited by 33
  3. 2019Towards More Accurate Automatic Sleep Staging via Deep Transfer Learningcited by 31
  4. 2016Deep Transfer Learning for Person Re-identificationcited by 16
  5. 2020Appropriate Learning Rates of Adaptive Learning Rate Optimization Algorithms for Training Deep Neural Networkscited by 16
  6. 2016ImageNet pre-trained models with batch normalizationcited by 14
  7. 2017Progressive Neural Networks for Transfer Learning in Emotion Recognitioncited by 12
  8. 2020Deep Transfer Learning for Signal Detection in Ambient Backscatter Communicationscited by 11
  9. 2017Understanding the Mechanisms of Deep Transfer Learning for Medical Imagescited by 10
  10. 2019Deep Learning at the Edgecited by 9
  11. 2021Deep Transfer Learning for WiFi Localizationcited by 7
  12. 2017Folding membrane proteins by deep transfer learningcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 12; 2024: 25; 2025: 10; 2026: 7.

56 papers.

Sort:

By area:

  1. 2023Efficient Adaptation of Large Vision Transformer via Adapter Re-Composingcited by 35
  2. 2023Learngene: Inheriting Condensed Knowledge from the Ancestry Model to Descendant Modelscited by 24
  3. 2024Low-Rank Rescaled Vision Transformer Fine-Tuning: A Residual Design Approachcited by 18
  4. 2024A Survey on Deep Tabular Learningcited by 18
  5. 2022Deep transfer learning for image classification: a surveycited by 13
  6. 2024CarDreamer: Open-Source Learning Platform for World Model based Autonomous Drivingcited by 13
  7. 2024Transferring Core Knowledge via Learngenescited by 10
  8. 2024Reconstructing Training Data From Real World Models Trained with Transfer Learningcited by 10
  9. 2023Genes in Intelligent Agentscited by 9
  10. 2023A Recent Survey of Heterogeneous Transfer Learningcited by 7
  11. 2024Bridging Data Gaps in Healthcare: A Scoping Review of Transfer Learning in Biomedical Data Analysiscited by 6
  12. 2024Efficient Adaptation of Pre-trained Vision Transformer via Householder Transformationcited by 5
38 more
  1. 2023Transfer learning for day-ahead load forecasting: a case study on European national electricity demand time seriescited by 4
  2. 2023How to Determine the Most Powerful Pre-trained Language Model without Brute Force Fine-tuning? An Empirical Surveycited by 4
  3. 2024Adapting WavLM for Speech Emotion Recognitioncited by 4
  4. 2023Can we learn better with hard samples?cited by 3
  5. 2024Transfer learning with generative models for object detection on limited datasetscited by 3
  6. 2024Computer Vision in the Food Industry: Accurate, Real-time, and Automatic Food Recognition with Pretrained MobileNetV2cited by 3
  7. 2024Federated and Transfer Learning for Cancer Detection Based on Image Analysiscited by 3
  8. 2024In-Context Transfer Learning: Demonstration Synthesis by Transferring Similar Taskscited by 3
  9. 2024What do physics-informed DeepONets learn? Understanding and improving training for scientific computing applicationscited by 3
  10. 2023Implicit regularisation in stochastic gradient descent: from single-objective to two-player gamescited by 2
  11. 2024Hyperdimensional computing: a fast, robust and interpretable paradigm for biological datacited by 2
  12. 2022Colab NAS: Obtaining lightweight task-specific convolutional neural networks following Occam's razorcited by 1
  13. 2023Mobile Traffic Prediction at the Edge Through Distributed and Deep Transfer Learningcited by 1
  14. 2023An Integrated Imitation and Reinforcement Learning Methodology for Robust Agile Aircraft Control with Limited Pilot Demonstration Datacited by 1
  15. 2024Online Transfer Learning for RSV Case Detectioncited by 1
  16. 2024Text2Data: Low-Resource Data Generation with Textual Controlcited by 1
  17. 2024Bias correction of wind power forecasts with SCADA data and continuous learningcited by 1
  18. 2024Data-driven Modeling in Metrology -- A Short Introduction, Current Developments and Future Perspectivescited by 1
  19. 2024Robust and Explainable Framework to Address Data Scarcity in Diagnostic Imagingcited by 1
  20. 2025Conditional Electrocardiogram Generation Using Hierarchical Variational Autoencoderscited by 1
  21. 2026Unsupervised Modular Adaptive Region Growing and RegionMix Classification for Wind Turbine Segmentationcited by 1
  22. 2026CBEN -- A Multimodal Machine Learning Dataset for Cloud Robust Remote Sensing Image Understandingcited by 1
  23. 2026Evolutionary Transfer Learning for Dragonchesscited by 1
  24. 2023On-device Real-time Custom Hand Gesture Recognitioncited by 0
  25. 2023Successive Model-Agnostic Meta-Learning for Few-Shot Fault Time Series Prognosiscited by 0
  26. 2024An active learning model to classify animal species in Hong Kongcited by 0
  27. 2024Leveraging Knowledge Distillation for Lightweight Skin Cancer Classification: Balancing Accuracy and Computational Efficiencycited by 0
  28. 2024A Novel Method for Accurate & Real-time Food Classification: The Synergistic Integration of EfficientNetB7, CBAM, Transfer Learning, and Data Augmentationcited by 0
  29. 2024Diagnosis of Malignant Lymphoma Cancer Using Hybrid Optimized Techniques Based on Dense Neural Networkscited by 0
  30. 2024SAMG: Offline-to-Online Reinforcement Learning via State-Action-Conditional Offline Model Guidancecited by 0
  31. 2024Deep Learning-Based Automatic Delineation of Liver Domes in kV Triggered Images for Online Breath-hold Reproducibility Verification of Liver Stereotactic Body Radiation Therapycited by 0
  32. 2025Fine-Tuning Federated Learning-Based Intrusion Detection Systems for Transportation IoTcited by 0
  33. 2025On the use of neural networks for the structural characterization of polymeric porous materialscited by 0
  34. 2025Acute Lymphoblastic Leukemia Diagnosis Employing YOLOv11, YOLOv8, ResNet50, and Inception-ResNet-v2 Deep Learning Modelscited by 0
  35. 202515,500 Seconds: Lean UAV Classification Using EfficientNet and Lightweight Fine-Tuningcited by 0
  36. 20253W Dataset 2.0.0: a realistic and public dataset with rare undesirable real events in oil wellscited by 0
  37. 2025Rethinking the Potential of Layer Freezing for Efficient DNN Trainingcited by 0
  38. 2025Fine-Tuned CNN-Based Approach for Multi-Class Mango Leaf Disease Detectioncited by 0

50 most cited of 56 All 56 (JSON)