mlsanityData to 7 Oct 2026

Sample Complexity of Transfer Learning: An Optimal Transport Approach

arXiv 2605.20545 ↗First posted 19 May 2026stat.ML cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Transfer learning is an essential technique for many machine learning/AI models of complex structures such as large language models and generative AI. The essence of transfer learning is to leverage knowledge from resolved source tasks for a new target task, especially when the sample size m of the training data for the latter is low. In this work, we rigorously analyze the potential benefit of transfer learning in terms of sample efficiency. Specifically, taking an optimal transport viewpoint of transfer learning, we find that when the data dimension d is higher than 3, the sample complexity for transfer learning is O(m-(α+1)/d), with α indicating the smoothness of the data distribution, as opposed to the O(m-p/d) sample complexity for direct learning with p indicating the smoothness of the optimal target model. Our finding theoretically supports a better sample efficiency for transfer learning, when the target task is optimizing over a family of not-so-smooth models (i.e., highly complex networks with the possible use of non-smooth activation functions). Using image classification as an example, we numerically demonstrate the sample efficiency for transfer learning, that is, in the data hungry regime, the model performance can be significantly improved by transfer learning.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 38 of its 74 references that are in the map. First posted on arXiv 19 May 2026.

Builds on

38 of 74 references.

References · 38

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  5. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,277
  6. 2021Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modelingcited by 594
  7. 2015Optimal Transport for Domain Adaptationcited by 457
  8. 2021VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Taskscited by 319
  9. 2020Few-Shot Learning via Learning the Representation, Provablycited by 213
  10. 2020On the Theory of Transfer Learning: The Importance of Task Diversitycited by 186
  11. 2020Provable Meta-Learning of Linear Representationscited by 161
  12. 2020Transfer Learning for High-dimensional Linear Regression: Prediction, Estimation, and Minimax Optimalitycited by 89
26 more
  1. 2015The Optimal Sample Complexity of PAC Learningcited by 72
  2. 2021Transfer Learning under High-dimensional Generalized Linear Modelscited by 69
  3. 2021Plugin Estimation of Smooth Optimal Transport Mapscited by 67
  4. 2019Transfer Learning for Nonparametric Classification: Minimax Rate and Adaptive Classifiercited by 64
  5. 2020On the Value of Target Data in Transfer Learningcited by 62
  6. 2013Sample Complexity of Multi-task Reinforcement Learningcited by 58
  7. 2024Statistical optimal transportcited by 44
  8. 2022Adaptive and Robust Multi-Task Learningcited by 33
  9. 2020A No-Free-Lunch Theorem for MultiTask Learningcited by 28
  10. 2023Learning from Similar Linear Representations: Adaptivity, Minimaxity, and Robustnesscited by 27
  11. 2022Transfer Learning for Contextual Multi-armed Banditscited by 24
  12. 2023On the Provable Advantage of Unsupervised Pretrainingcited by 21
  13. 2024TransFusion: Covariate-Shift Robust Transfer Learning for High-Dimensional Regressioncited by 11
  14. 2024Features are fate: a theory of transfer learning in high-dimensional regressioncited by 10
  15. 2023Limits of Model Selection under Transfer Learningcited by 9
  16. 2021An Optimal Transport Approach to Estimating Causal Effects via Nonlinear Difference-in-Differencescited by 7
  17. 2018Optimal transportation between unequal dimensionscited by 6
  18. 2025Provable Benefits of Unsupervised Pre-training and Transfer Learning via Single-Index Modelscited by 6
  19. 2022Transfer Learning for Retinal Vascular Disease Detection: A Pilot Study with Diabetic Retinopathy and Retinopathy of Prematuritycited by 5
  20. 2022On the Sample Complexity of Representation Learning in Multi-task Bandits with Global and Local structurecited by 5
  21. 2023Risk of Transfer Learning and its Applications in Financecited by 5
  22. 2025Provable Sample-Efficient Transfer Learning Conditional Diffusion Models via Representation Learningcited by 5
  23. 2023Robust Transfer Learning with Unreliable Source Datacited by 4
  24. 2025Partial Domain Adaptation via Importance Sampling-based Shift Correctioncited by 2
  25. 2025A Unified Analysis of Generalization and Sample Complexity for Semi-Supervised Domain Adaptationcited by 1
  26. 2025IBEX: Information-Bottleneck-EXplored Coarse-to-Fine Molecular Generation under Limited Datacited by 1

Built on by

Not cited yet.