# Heterogeneous Transfer Learning for Building High-Dimensional Generalized Linear Models with Disparate Datasets

arXiv: https://arxiv.org/abs/2312.12786 · HTML: https://mlsanity.com/paper/2312.12786/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 66 references that are in the map.
- First posted on arXiv 20 Dec 2023.

## Builds on (most cited first, 12 of 12)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Transfusion: Understanding Transfer Learning for Medical Imaging](https://mlsanity.com/paper/1902.07208/) (2019)
- [Transfer Learning in Deep Reinforcement Learning: A Survey](https://mlsanity.com/paper/2009.07888/) (2020)
- [Transfer Learning for High-dimensional Linear Regression: Prediction, Estimation, and Minimax Optimality](https://mlsanity.com/paper/2006.10593/) (2020)
- [BAM! Born-Again Multi-Task Networks for Natural Language Understanding](https://mlsanity.com/paper/1907.04829/) (2019)
- [Transfer Learning under High-dimensional Generalized Linear Models](https://mlsanity.com/paper/2105.14328/) (2021)
- [The Variational Method of Moments](https://mlsanity.com/paper/2012.09422/) (2020)
- [Heterogeneity-aware and communication-efficient distributed statistical inference](https://mlsanity.com/paper/1912.09623/) (2019)
- [An Efficient Method of Training Small Models for Regression Problems with Knowledge Distillation](https://mlsanity.com/paper/2002.12597/) (2020)
- [Generalized Meta-Analysis for Multiple Regression Models Across Studies with Disparate Covariate Information](https://mlsanity.com/paper/1708.03818/) (2017)
- [Inference in High Dimensions with the Penalized Score Test](https://mlsanity.com/paper/1401.2678/) (2014)
- [A meta-inference framework to integrate multiple external models into a current study](https://mlsanity.com/paper/2010.09971/) (2020)

## Built on by (most cited first, 1 of 1)

- [Hierarchical Projection for Adaptive Knowledge Transfer](https://mlsanity.com/paper/2606.08691/) (2026)

## Abstract (arXiv)

Development of comprehensive prediction models are often of great interest in many disciplines of science, but datasets with information on all desired features often have small sample sizes. We describe a transfer learning approach for building high-dimensional generalized linear models using data from a main study with detailed information on all predictors and an external, potentially much larger, study that has ascertained a more limited set of predictors. We propose using the external dataset to build a reduced model and then "transfer" the information on underlying parameters for the analysis of the main study through a set of calibration equations which can account for the study-specific effects of design variables. We then propose a penalized generalized method of moment framework for inference and a one-step estimation method that could be implemented using standard glmnet package. We develop asymptotic theory and conduct extensive simulation studies to investigate both predictive performance and post-selection inference properties of the proposed method. Finally, we illustrate an application of the proposed method for the development of risk models for five common diseases using the UK Biobank study, combining information on low-dimensional risk factors and high throughout proteomic biomarkers.