# Feasibility of Transfer Learning: A Mathematical Framework

arXiv: https://arxiv.org/abs/2305.12985 · HTML: https://mlsanity.com/paper/2305.12985/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 53 references that are in the map.
- First posted on arXiv 22 May 2023.

## Builds on (most cited first, 30 of 30)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [Deep Visual Domain Adaptation: A Survey](https://mlsanity.com/paper/1802.03601/) (2018)
- [Optimal Transport for Domain Adaptation](https://mlsanity.com/paper/1507.00504/) (2015)
- [A Survey on Deep Transfer Learning](https://mlsanity.com/paper/1808.01974/) (2018)
- [VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Tasks](https://mlsanity.com/paper/2112.06825/) (2021)
- [LEEP: A New Measure to Evaluate Transferability of Learned Representations](https://mlsanity.com/paper/2002.12462/) (2020)
- [On the Theory of Transfer Learning: The Importance of Task Diversity](https://mlsanity.com/paper/2006.11650/) (2020)
- [LogME: Practical Assessment of Pre-trained Models for Transfer Learning](https://mlsanity.com/paper/2102.11005/) (2021)
- [Structured Pruning Learns Compact and Accurate Models](https://mlsanity.com/paper/2204.00408/) (2022)
- [Transferability and Hardness of Supervised Classification Tasks](https://mlsanity.com/paper/1908.08142/) (2019)
- [Regularized Learning for Domain Adaptation under Label Shifts](https://mlsanity.com/paper/1903.09734/) (2019)
- [Tightening Mutual Information Based Bounds on Generalization Error](https://mlsanity.com/paper/1901.04609/) (2019)
- [Deep Visual Domain Adaptation](https://mlsanity.com/paper/2012.14176/) (2020)
- [An Information-Theoretic Approach to Transferability in Task Transfer Learning](https://mlsanity.com/paper/2212.10082/) (2022)
- [Multi-level Semantic Feature Augmentation for One-shot Learning](https://mlsanity.com/paper/1804.05298/) (2018)
- [OTCE: A Transferability Metric for Cross-Domain Cross-Task Representations](https://mlsanity.com/paper/2103.13843/) (2021)
- [Transferability Estimation using Bhattacharyya Class Separability](https://mlsanity.com/paper/2111.12780/) (2021)
- [Adversarial Feature Augmentation for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1711.08561/) (2017)
- [Frustratingly Easy Transferability Estimation](https://mlsanity.com/paper/2106.09362/) (2021)
- [Stratified Transfer Learning for Cross-domain Activity Recognition](https://mlsanity.com/paper/1801.00820/) (2017)
- [DARec: Deep Domain Adaptation for Cross-Domain Recommendation via Transferring Rating Patterns](https://mlsanity.com/paper/1905.10760/) (2019)
- [Minimax Lower Bounds for Transfer Learning with Linear and One-hidden Layer Neural Networks](https://mlsanity.com/paper/2006.10581/) (2020)
- [Learning a functional control for high-frequency finance](https://mlsanity.com/paper/2006.09611/) (2020)
- [Transfer Learning for Retinal Vascular Disease Detection: A Pilot Study with Diabetic Retinopathy and Retinopathy of Prematurity](https://mlsanity.com/paper/2201.01250/) (2022)
- [Generalization Bounds for Deep Transfer Learning Using Majority Predictor Accuracy](https://mlsanity.com/paper/2209.05709/) (2022)

## Built on by (most cited first, 2 of 2)

- [Fast Policy Learning for Linear Quadratic Control with Entropy Regularization](https://mlsanity.com/paper/2311.14168/) (2023)
- [Transport-Related Surface Detection with Machine Learning: Analyzing Temporal Trends in Madrid and Vienna](https://mlsanity.com/paper/2503.15653/) (2025)

## Abstract (arXiv)

Transfer learning is a popular paradigm for utilizing existing knowledge from previous learning tasks to improve the performance of new ones. It has enjoyed numerous empirical successes and inspired a growing number of theoretical studies. This paper addresses the feasibility issue of transfer learning. It begins by establishing the necessary mathematical concepts and constructing a mathematical framework for transfer learning. It then identifies and formulates the three-step transfer learning procedure as an optimization problem, allowing for the resolution of the feasibility issue. Importantly, it demonstrates that under certain technical conditions, such as appropriate choice of loss functions and data sets, an optimal procedure for transfer learning exists. This study of the feasibility issue brings additional insights into various transfer learning problems. It sheds light on the impact of feature augmentation on model performance, explores potential extensions of domain adaptation, and examines the feasibility of efficient feature extractor transfer in image classification.