# Transfer Learning under High-dimensional Generalized Linear Models

arXiv: https://arxiv.org/abs/2105.14328 · HTML: https://mlsanity.com/paper/2105.14328/

## Key facts

- Cited by 70 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 47 references that are in the map.
- First posted on arXiv 29 May 2021.

## Builds on (most cited first, 11 of 11)

- [Transfer Learning for High-dimensional Linear Regression: Prediction, Estimation, and Minimax Optimality](https://mlsanity.com/paper/2006.10593/) (2020)
- [Regularized M-estimators with nonconvexity: Statistical and algorithmic theory for local optima](https://mlsanity.com/paper/1305.2436/) (2013)
- [Transfer Learning for Nonparametric Classification: Minimax Rate and Adaptive Classifier](https://mlsanity.com/paper/1906.02903/) (2019)
- [On the Value of Target Data in Transfer Learning](https://mlsanity.com/paper/2002.04747/) (2020)
- [Transfer Learning in Large-scale Gaussian Graphical Models with False Discovery Rate Control](https://mlsanity.com/paper/2010.11037/) (2020)
- [A No-Free-Lunch Theorem for MultiTask Learning](https://mlsanity.com/paper/2006.15785/) (2020)
- [Predicting with Proxies: Transfer Learning in High Dimension](https://mlsanity.com/paper/1812.11097/) (2018)
- [Concentration in unbounded metric spaces and algorithmic stability](https://mlsanity.com/paper/1309.1007/) (2013)
- [Bayesian multi-domain learning for cancer subtype discovery from next-generation sequencing count data](https://mlsanity.com/paper/1810.09433/) (2018)
- [Data enriched linear regression](https://mlsanity.com/paper/1304.1837/) (2013)
- [Regression modeling on stratified data with the lasso](https://mlsanity.com/paper/1508.05476/) (2015)

## Built on by (most cited first, 30 of 70)

- [Prediction-Powered Inference](https://mlsanity.com/paper/2301.09633/) (2023)
- [Adaptive and Robust Multi-Task Learning](https://mlsanity.com/paper/2202.05250/) (2022)
- [Targeting Underrepresented Populations in Precision Medicine: A Federated Transfer Learning Approach](https://mlsanity.com/paper/2108.12112/) (2021)
- [Multitask Learning and Bandits via Robust Statistics](https://mlsanity.com/paper/2112.14233/) (2021)
- [TransFusion: Covariate-Shift Robust Transfer Learning for High-Dimensional Regression](https://mlsanity.com/paper/2404.01153/) (2024)
- [A Class of Geometric Structures in Transfer Learning: Minimax Bounds and Optimality](https://mlsanity.com/paper/2202.11685/) (2022)
- [Smoothness Adaptive Hypothesis Transfer Learning](https://mlsanity.com/paper/2402.14966/) (2024)
- [Optimal Multitask Linear Regression and Contextual Bandits under Sparse Heterogeneity](https://mlsanity.com/paper/2306.06291/) (2023)
- [Transfer Learning with Uncertainty Quantification: Random Effect Calibration of Source to Target (RECaST)](https://mlsanity.com/paper/2211.16557/) (2022)
- [Efficient and Multiply Robust Risk Estimation under General Forms of Dataset Shift](https://mlsanity.com/paper/2306.16406/) (2023)
- [Representation Retrieval Learning for Heterogeneous Data Integration](https://mlsanity.com/paper/2503.09494/) (2025)
- [Generalization Performance of Transfer Learning: Overparameterized and Underparameterized Regimes](https://mlsanity.com/paper/2306.04901/) (2023)
- [Guided Adversarial Robust Transfer Learning with Source Mixing](https://mlsanity.com/paper/2309.06534/) (2023)
- [Unified Transfer Learning Models in High-Dimensional Linear Regression](https://mlsanity.com/paper/2307.00238/) (2023)
- [Transfer Learning for Kernel-based Regression](https://mlsanity.com/paper/2310.13966/) (2023)
- [AdaTrans: Feature-wise and Sample-wise Adaptive Transfer Learning for High-dimensional Regression](https://mlsanity.com/paper/2403.13565/) (2024)
- [Distributionally robust risk evaluation with an isotonic constraint](https://mlsanity.com/paper/2407.06867/) (2024)
- [LLM Embeddings Improve Test-time Adaptation to Tabular Y|X-Shifts](https://mlsanity.com/paper/2410.07395/) (2024)
- [Transfer Learning for Benign Overfitting in High-Dimensional Linear Regression](https://mlsanity.com/paper/2510.15337/) (2025)
- [On Hypothesis Transfer Learning of Functional Linear Models](https://mlsanity.com/paper/2206.04277/) (2022)
- [Transfer learning for tensor Gaussian graphical models](https://mlsanity.com/paper/2211.09391/) (2022)
- [Multi-dimensional domain generalization with low-rank structures](https://mlsanity.com/paper/2309.09555/) (2023)
- [Covariate-Elaborated Robust Partial Information Transfer with Conditional Spike-and-Slab Prior](https://mlsanity.com/paper/2404.03764/) (2024)
- [Trans-Glasso: A Transfer Learning Approach to Precision Matrix Estimation](https://mlsanity.com/paper/2411.15624/) (2024)
- [Fixed-Gaussian Spectral Algorithms: Minimax Optimal Rates for Misspecified Learning and Transfer](https://mlsanity.com/paper/2501.10870/) (2025)
- [Wasserstein Transfer Learning](https://mlsanity.com/paper/2505.17404/) (2025)
- [Late Fusion Multi-task Learning for Semiparametric Inference with Nuisance Parameters](https://mlsanity.com/paper/2507.07941/) (2025)
- [The ART of Transfer Learning: An Adaptive and Robust Pipeline](https://mlsanity.com/paper/2305.00520/) (2023)
- [Heterogeneous Transfer Learning for Building High-Dimensional Generalized Linear Models with Disparate Datasets](https://mlsanity.com/paper/2312.12786/) (2023)
- [Transfer Learning for Spatial Autoregressive Models with Application to U.S. Presidential Election Prediction](https://mlsanity.com/paper/2405.15600/) (2024)

## Abstract (arXiv)

In this work, we study the transfer learning problem under high-dimensional generalized linear models (GLMs), which aim to improve the fit on target data by borrowing information from useful source data. Given which sources to transfer, we propose a transfer learning algorithm on GLM, and derive its ℓ1/ℓ2-estimation error bounds as well as a bound for a prediction error measure. The theoretical analysis shows that when the target and source are sufficiently close to each other, these bounds could be improved over those of the classical penalized estimator using only target data under mild conditions. When we don't know which sources to transfer, an algorithm-free transferable source detection approach is introduced to detect informative sources. The detection consistency is proved under the high-dimensional GLM transfer learning setting. We also propose an algorithm to construct confidence intervals of each coefficient component, and the corresponding theories are provided. Extensive simulations and a real-data experiment verify the effectiveness of our algorithms. We implement the proposed GLM transfer learning algorithms in a new R package glmtrans, which is available on CRAN.