# Heterogeneous transfer learning for high-dimensional regression with feature mismatch

arXiv: https://arxiv.org/abs/2412.18081 · HTML: https://mlsanity.com/paper/2412.18081/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Dec 2024.

## Built on by (most cited first, 5 of 5)

- [Representation Retrieval Learning for Heterogeneous Data Integration](https://mlsanity.com/paper/2503.09494/) (2025)
- [Transfer Learning with Distance Covariance for Random Forest: Error Bounds and an EHR Application](https://mlsanity.com/paper/2510.10870/) (2025)
- [Hierarchical Projection for Adaptive Knowledge Transfer](https://mlsanity.com/paper/2606.08691/) (2026)
- [Black-Box Knowledge Transfer across Distinct Feature Sets](https://mlsanity.com/paper/2608.12403/) (2026)
- [Transfer Calibrated Prediction Powered Inference](https://mlsanity.com/paper/2609.34156/) (2026)

## Abstract (arXiv)

We study Heterogeneous Transfer Learning (HTL) for high-dimensional regression with differing feature sets. Such feature mismatch arises when some variables available in a data-rich source domain are unavailable in a data-poor target domain. Yet most homogeneous TL methods require the same feature space in both the source and target domains, limiting their practical applicability. Conversely, existing HTL methods lack statistical error guarantees, limiting their utility for scientific discovery. We propose an HTL method that first learns a feature map between the missing and observed features leveraging the vast source data, imputes the unavailable features in the target, and then performs a two-step TL for penalized regression. We consider both the linear and the nonparametric feature maps. We develop upper bounds on the estimation and prediction errors of HTL, assuming that the source and target parameters differ sparsely, without requiring the target model itself to be sparse. We also establish matching minimax lower bounds, showing that the proposed procedures achieve optimal rates. Our results elucidate the effects of model complexity, sample size, the quality and differences in feature maps, and differences in the models across domains. We also derive minimax rates for the misspecified homogeneous TL model that discards unavailable features and show that our HTL procedure can attain a smaller error rate than homogeneous TL. We further extend the framework to multiple source domains and develop a negative-transfer defense that provably excludes adversarial sources from transfer with high probability.