mlsanityData to 7 Oct 2026

Feasibility and Transferability of Transfer Learning: A Mathematical Framework

arXiv 2301.11542 ↗First posted 27 Jan 2023cs.LG stat.MLcited by 2

Not on the map. Why?

Abstract (arXiv)

Transfer learning is an emerging and popular paradigm for utilizing existing knowledge from previous learning tasks to improve the performance of new ones. Despite its numerous empirical successes, theoretical analysis for transfer learning is limited. In this paper we build for the first time, to the best of our knowledge, a mathematical framework for the general procedure of transfer learning. Our unique reformulation of transfer learning as an optimization problem allows for the first time, analysis of its feasibility. Additionally, we propose a novel concept of transfer risk to evaluate transferability of transfer learning. Our numerical studies using the Office-31 dataset demonstrate the potential and benefits of incorporating transfer risk in the evaluation of transfer learning performance.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 18 of its 39 references that are in the map. First posted on arXiv 27 Jan 2023.

Builds on

18 of 39 references.

References · 18

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  3. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,906
  4. 2015Learning Transferable Features with Deep Adaptation Networkscited by 1,323
  5. 2017Adversarial Discriminative Domain Adaptationcited by 1,309
  6. 2019A Comprehensive Survey on Transfer Learningcited by 1,154
  7. 2018Deep Visual Domain Adaptation: A Surveycited by 634
  8. 2015Optimal Transport for Domain Adaptationcited by 457
  9. 2018A Survey on Deep Transfer Learningcited by 417
  10. 2021VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Taskscited by 319
  11. 2022Structured Pruning Learns Compact and Accurate Modelscited by 167
  12. 2019Tightening Mutual Information Based Bounds on Generalization Errorcited by 117
6 more
  1. 2022An Information-Theoretic Approach to Transferability in Task Transfer Learningcited by 79
  2. 2021OTCE: A Transferability Metric for Cross-Domain Cross-Task Representationscited by 62
  3. 2017Stratified Transfer Learning for Cross-domain Activity Recognitioncited by 40
  4. 2019DARec: Deep Domain Adaptation for Cross-Domain Recommendation via Transferring Rating Patternscited by 33
  5. 2020Learning a functional control for high-frequency financecited by 9
  6. 2022Transfer Learning for Retinal Vascular Disease Detection: A Pilot Study with Diabetic Retinopathy and Retinopathy of Prematuritycited by 5

Built on by

Cited by year

Citing papers per year: 2023: 2.

2 papers.

  1. 2023Risk of Transfer Learning and its Applications in Financecited by 5
  2. 2023Transfer Learning for Portfolio Optimizationcited by 1