Feasibility of Transfer Learning: A Mathematical Framework
Not on the map. Why?
Abstract (arXiv)
Transfer learning is a popular paradigm for utilizing existing knowledge from previous learning tasks to improve the performance of new ones. It has enjoyed numerous empirical successes and inspired a growing number of theoretical studies. This paper addresses the feasibility issue of transfer learning. It begins by establishing the necessary mathematical concepts and constructing a mathematical framework for transfer learning. It then identifies and formulates the three-step transfer learning procedure as an optimization problem, allowing for the resolution of the feasibility issue. Importantly, it demonstrates that under certain technical conditions, such as appropriate choice of loss functions and data sets, an optimal procedure for transfer learning exists. This study of the feasibility issue brings additional insights into various transfer learning problems. It sheds light on the impact of feature augmentation on model performance, explores potential extensions of domain adaptation, and examines the feasibility of efficient feature extractor transfer in image classification.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 30 of its 53 references that are in the map. First posted on arXiv 22 May 2023.
Builds on
30 of 53 references.
References · 30
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2015Domain-Adversarial Training of Neural Networkscited by 3,023
- 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,906
- 2015Learning Transferable Features with Deep Adaptation Networkscited by 1,323
- 2017Adversarial Discriminative Domain Adaptationcited by 1,309
- 2019A Comprehensive Survey on Transfer Learningcited by 1,154
- 2018Deep Visual Domain Adaptation: A Surveycited by 634
- 2015Optimal Transport for Domain Adaptationcited by 457
- 2018A Survey on Deep Transfer Learningcited by 417
- 2021VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Taskscited by 319
- 2020On the Theory of Transfer Learning: The Importance of Task Diversitycited by 186
- 2020LEEP: A New Measure to Evaluate Transferability of Learned Representationscited by 182
18 more
- 2021LogME: Practical Assessment of Pre-trained Models for Transfer Learningcited by 168
- 2022Structured Pruning Learns Compact and Accurate Modelscited by 167
- 2019Transferability and Hardness of Supervised Classification Taskscited by 125
- 2019Regularized Learning for Domain Adaptation under Label Shiftscited by 118
- 2019Tightening Mutual Information Based Bounds on Generalization Errorcited by 117
- 2020Deep Visual Domain Adaptationcited by 91
- 2022An Information-Theoretic Approach to Transferability in Task Transfer Learningcited by 79
- 2018Multi-level Semantic Feature Augmentation for One-shot Learningcited by 62
- 2021OTCE: A Transferability Metric for Cross-Domain Cross-Task Representationscited by 62
- 2021Transferability Estimation using Bhattacharyya Class Separabilitycited by 53
- 2017Adversarial Feature Augmentation for Unsupervised Domain Adaptationcited by 48
- 2021Frustratingly Easy Transferability Estimationcited by 44
- 2017Stratified Transfer Learning for Cross-domain Activity Recognitioncited by 40
- 2019DARec: Deep Domain Adaptation for Cross-Domain Recommendation via Transferring Rating Patternscited by 33
- 2020Minimax Lower Bounds for Transfer Learning with Linear and One-hidden Layer Neural Networkscited by 30
- 2020Learning a functional control for high-frequency financecited by 9
- 2022Transfer Learning for Retinal Vascular Disease Detection: A Pilot Study with Diabetic Retinopathy and Retinopathy of Prematuritycited by 5
- 2022Generalization Bounds for Deep Transfer Learning Using Majority Predictor Accuracycited by 4
Built on by
Citing papers per year: 2023: 1; 2024: 0; 2025: 1.
2 papers.