# A High-Dimensional Statistical Method for Optimizing Transfer Quantities in Multi-Source Transfer Learning

arXiv: https://arxiv.org/abs/2502.04242 · HTML: https://mlsanity.com/paper/2502.04242/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 38 references that are in the map.
- First posted on arXiv 6 Feb 2025.

## Builds on (most cited first, 23 of 23)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [Deep Visual Domain Adaptation: A Survey](https://mlsanity.com/paper/1802.03601/) (2018)
- [Optimal Transport for Domain Adaptation](https://mlsanity.com/paper/1507.00504/) (2015)
- [New insights and perspectives on the natural gradient method](https://mlsanity.com/paper/1412.1193/) (2014)
- [Transfusion: Understanding Transfer Learning for Medical Imaging](https://mlsanity.com/paper/1902.07208/) (2019)
- [VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Tasks](https://mlsanity.com/paper/2112.06825/) (2021)
- [LEEP: A New Measure to Evaluate Transferability of Learned Representations](https://mlsanity.com/paper/2002.12462/) (2020)
- [Tightening Mutual Information Based Bounds on Generalization Error](https://mlsanity.com/paper/1901.04609/) (2019)
- [A Review of Single-Source Deep Unsupervised Visual Domain Adaptation](https://mlsanity.com/paper/2009.00155/) (2020)
- [An Information-Theoretic Approach to Transferability in Task Transfer Learning](https://mlsanity.com/paper/2212.10082/) (2022)
- [Multi-Source Domain Adaptation for Text Classification via DistanceNet-Bandits](https://mlsanity.com/paper/2001.04362/) (2020)
- [Dynamic Transfer for Multi-Source Domain Adaptation](https://mlsanity.com/paper/2103.10583/) (2021)
- [Exploring Adapter-based Transfer Learning for Recommender Systems: Empirical Studies and Practical Insights](https://mlsanity.com/paper/2305.15036/) (2023)
- [Aggregating From Multiple Target-Shifted Sources](https://mlsanity.com/paper/2105.04051/) (2021)
- [PipeFisher: Efficient Training of Large Language Models Using Pipelining and Fisher Information Matrices](https://mlsanity.com/paper/2211.14133/) (2022)
- [MADAN: Multi-source Adversarial Domain Aggregation Network for Domain Adaptation](https://mlsanity.com/paper/2003.00820/) (2020)
- [Revisiting the Domain Shift and Sample Uncertainty in Multi-source Active Domain Transfer](https://mlsanity.com/paper/2311.12905/) (2023)
- [On Balancing Bias and Variance in Unsupervised Multi-Source-Free Domain Adaptation](https://mlsanity.com/paper/2202.00796/) (2022)
- [More is Better: Deep Domain Adaptation with Multiple Sources](https://mlsanity.com/paper/2405.00749/) (2024)
- [H-ensemble: An Information Theoretic Approach to Reliable Few-Shot Multi-Source-Free Transfer](https://mlsanity.com/paper/2312.12489/) (2023)

## Built on by (most cited first, 5 of 5)

- [LoRA-DA: Data-Aware Initialization for Low-Rank Adaptation via Asymptotic Analysis](https://mlsanity.com/paper/2510.24561/) (2025)
- [Exposure Bias Can Alleviate Itself via Directional and Frequency Rectification in Flow Matching](https://mlsanity.com/paper/2606.28226/) (2026)
- [Unified Optimization of Source Weights and Transfer Quantities in Multi-Source Transfer Learning: An Asymptotic Framework](https://mlsanity.com/paper/2601.10779/) (2026)
- [OSDTW: Optimal Shared Depth and Task Weighting for Long-Tailed Recognition](https://mlsanity.com/paper/2605.24969/) (2026)
- [Beyond Non-IID: Learner--Client Distribution Mismatch in Federated Learning](https://mlsanity.com/paper/2608.27715/) (2026)

## Abstract (arXiv)

Multi-source transfer learning provides an effective solution to data scarcity in real-world supervised learning scenarios by leveraging multiple source tasks. In this field, existing works typically use all available samples from sources in training, which constrains their training efficiency and may lead to suboptimal results. To address this, we propose a theoretical framework that answers the question: what is the optimal quantity of source samples needed from each source task to jointly train the target model? Specifically, we introduce a generalization error measure based on K-L divergence, and minimize it based on high-dimensional statistical analysis to determine the optimal transfer quantity for each source task. Additionally, we develop an architecture-agnostic and data-efficient algorithm OTQMS to implement our theoretical results for target model training in multi-source transfer learning. Experimental studies on diverse architectures and two real-world benchmark datasets show that our proposed algorithm significantly outperforms state-of-the-art approaches in both accuracy and data efficiency. The code and supplementary materials are available in https://github.com/zqy0126/OTQMS.