# Transfer Learning for High-dimensional Linear Regression: Prediction, Estimation, and Minimax Optimality

arXiv: https://arxiv.org/abs/2006.10593 · HTML: https://mlsanity.com/paper/2006.10593/

## Key facts

- Cited by 90 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jun 2020.

## Built on by (most cited first, 30 of 90)

- [Transfer Learning under High-dimensional Generalized Linear Models](https://mlsanity.com/paper/2105.14328/) (2021)
- [Adaptive and Robust Multi-Task Learning](https://mlsanity.com/paper/2202.05250/) (2022)
- [Transfer Learning for Contextual Multi-armed Bandits](https://mlsanity.com/paper/2211.12612/) (2022)
- [Transfer Learning for Nonparametric Regression: Non-asymptotic Minimax Analysis and Adaptive Procedure](https://mlsanity.com/paper/2401.12272/) (2024)
- [Targeting Underrepresented Populations in Precision Medicine: A Federated Transfer Learning Approach](https://mlsanity.com/paper/2108.12112/) (2021)
- [Multitask Learning and Bandits via Robust Statistics](https://mlsanity.com/paper/2112.14233/) (2021)
- [TransFusion: Covariate-Shift Robust Transfer Learning for High-Dimensional Regression](https://mlsanity.com/paper/2404.01153/) (2024)
- [A Class of Geometric Structures in Transfer Learning: Minimax Bounds and Optimality](https://mlsanity.com/paper/2202.11685/) (2022)
- [Smoothness Adaptive Hypothesis Transfer Learning](https://mlsanity.com/paper/2402.14966/) (2024)
- [Analysing Multi-Task Regression via Random Matrix Theory with Application to Time Series Forecasting](https://mlsanity.com/paper/2406.10327/) (2024)
- [Optimal Multitask Linear Regression and Contextual Bandits under Sparse Heterogeneity](https://mlsanity.com/paper/2306.06291/) (2023)
- [Towards the Theory of Unsupervised Federated Learning: Non-asymptotic Analysis of Federated EM Algorithms](https://mlsanity.com/paper/2310.15330/) (2023)
- [Minimax Regret Learning for Data with Heterogeneous Subgroups](https://mlsanity.com/paper/2405.01709/) (2024)
- [Transfer Learning with Uncertainty Quantification: Random Effect Calibration of Source to Target (RECaST)](https://mlsanity.com/paper/2211.16557/) (2022)
- [What can we learn from quantum convolutional neural networks?](https://mlsanity.com/paper/2308.16664/) (2023)
- [Bridging Data Gaps in Healthcare: A Scoping Review of Transfer Learning in Biomedical Data Analysis](https://mlsanity.com/paper/2407.11034/) (2024)
- [Deep Transfer Learning: Model Framework and Error Analysis](https://mlsanity.com/paper/2410.09383/) (2024)
- [Representation Retrieval Learning for Heterogeneous Data Integration](https://mlsanity.com/paper/2503.09494/) (2025)
- [Generalization Performance of Transfer Learning: Overparameterized and Underparameterized Regimes](https://mlsanity.com/paper/2306.04901/) (2023)
- [Guided Adversarial Robust Transfer Learning with Source Mixing](https://mlsanity.com/paper/2309.06534/) (2023)
- [Transfer Q-learning](https://mlsanity.com/paper/2202.04709/) (2022)
- [Unified Transfer Learning Models in High-Dimensional Linear Regression](https://mlsanity.com/paper/2307.00238/) (2023)
- [Transfer Learning for Kernel-based Regression](https://mlsanity.com/paper/2310.13966/) (2023)
- [AdaTrans: Feature-wise and Sample-wise Adaptive Transfer Learning for High-dimensional Regression](https://mlsanity.com/paper/2403.13565/) (2024)
- [Distributionally robust risk evaluation with an isotonic constraint](https://mlsanity.com/paper/2407.06867/) (2024)
- [LLM Embeddings Improve Test-time Adaptation to Tabular Y|X-Shifts](https://mlsanity.com/paper/2410.07395/) (2024)
- [Transfer Learning for Benign Overfitting in High-Dimensional Linear Regression](https://mlsanity.com/paper/2510.15337/) (2025)
- [On Hypothesis Transfer Learning of Functional Linear Models](https://mlsanity.com/paper/2206.04277/) (2022)
- [Transfer learning for tensor Gaussian graphical models](https://mlsanity.com/paper/2211.09391/) (2022)
- [Transfer Learning with Random Coefficient Ridge Regression](https://mlsanity.com/paper/2306.15915/) (2023)

## Abstract (arXiv)

This paper considers the estimation and prediction of a high-dimensional linear regression in the setting of transfer learning, using samples from the target model as well as auxiliary samples from different but possibly related regression models. When the set of "informative" auxiliary samples is known, an estimator and a predictor are proposed and their optimality is established. The optimal rates of convergence for prediction and estimation are faster than the corresponding rates without using the auxiliary samples. This implies that knowledge from the informative auxiliary samples can be transferred to improve the learning performance of the target problem. In the case that the set of informative auxiliary samples is unknown, we propose a data-driven procedure for transfer learning, called Trans-Lasso, and reveal its robustness to non-informative auxiliary samples and its efficiency in knowledge transfer. The proposed procedures are demonstrated in numerical studies and are applied to a dataset concerning the associations among gene expressions. It is shown that Trans-Lasso leads to improved performance in gene expression prediction in a target tissue by incorporating the data from multiple different tissues as auxiliary samples.