# TransFusion: Covariate-Shift Robust Transfer Learning for High-Dimensional Regression

arXiv: https://arxiv.org/abs/2404.01153 · HTML: https://mlsanity.com/paper/2404.01153/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 44 references that are in the map.
- First posted on arXiv 1 Apr 2024.

## Builds on (most cited first, 14 of 14)

- [Few-Shot Learning via Learning the Representation, Provably](https://mlsanity.com/paper/2002.09434/) (2020)
- [On the Theory of Transfer Learning: The Importance of Task Diversity](https://mlsanity.com/paper/2006.11650/) (2020)
- [MNIST-C: A Robustness Benchmark for Computer Vision](https://mlsanity.com/paper/1906.02337/) (2019)
- [A survey on domain adaptation theory: learning bounds and theoretical guarantees](https://mlsanity.com/paper/2004.11829/) (2020)
- [Transfer Learning for High-dimensional Linear Regression: Prediction, Estimation, and Minimax Optimality](https://mlsanity.com/paper/2006.10593/) (2020)
- [Confidence Intervals and Hypothesis Testing for High-Dimensional Regression](https://mlsanity.com/paper/1306.3171/) (2013)
- [Transfer Learning under High-dimensional Generalized Linear Models](https://mlsanity.com/paper/2105.14328/) (2021)
- [Near-Optimal Linear Regression under Distribution Shift](https://mlsanity.com/paper/2106.12108/) (2021)
- [Learning from Similar Linear Representations: Adaptivity, Minimaxity, and Robustness](https://mlsanity.com/paper/2303.17765/) (2023)
- [Hypothesis Transfer Learning via Transformation Functions](https://mlsanity.com/paper/1612.01020/) (2016)
- [A highly efficient semismooth Newton augmented Lagrangian method for solving Lasso problems](https://mlsanity.com/paper/1607.05428/) (2016)
- [Smoothness Adaptive Hypothesis Transfer Learning](https://mlsanity.com/paper/2402.14966/) (2024)
- [Optimal Multitask Linear Regression and Contextual Bandits under Sparse Heterogeneity](https://mlsanity.com/paper/2306.06291/) (2023)
- [Unified Transfer Learning Models in High-Dimensional Linear Regression](https://mlsanity.com/paper/2307.00238/) (2023)

## Built on by (most cited first, 11 of 11)

- [Transfer Learning for Benign Overfitting in High-Dimensional Linear Regression](https://mlsanity.com/paper/2510.15337/) (2025)
- [Trans-Glasso: A Transfer Learning Approach to Precision Matrix Estimation](https://mlsanity.com/paper/2411.15624/) (2024)
- [Understanding the Statistical Accuracy-Communication Trade-off in Personalized Federated Learning with Minimax Guarantees](https://mlsanity.com/paper/2410.08934/) (2024)
- [Multi-task Learning for Heterogeneous Multi-source Block-Wise Missing Data](https://mlsanity.com/paper/2505.24413/) (2025)
- [Minimum Wasserstein distance estimator under covariate shift: closed-form, super-efficiency and irregularity](https://mlsanity.com/paper/2601.07282/) (2026)
- [Post-Transfer Learning Statistical Inference in High-Dimensional Regression](https://mlsanity.com/paper/2504.18212/) (2025)
- [Transfer Learning for Matrix Completion](https://mlsanity.com/paper/2507.02248/) (2025)
- [Representation-Aware Distributionally Robust Optimization: A Knowledge Transfer Framework](https://mlsanity.com/paper/2509.09371/) (2025)
- [Transfer learning under latent space model](https://mlsanity.com/paper/2509.15797/) (2025)
- [Transfer Learning with Distance Covariance for Random Forest: Error Bounds and an EHR Application](https://mlsanity.com/paper/2510.10870/) (2025)
- [Sample Complexity of Transfer Learning: An Optimal Transport Approach](https://mlsanity.com/paper/2605.20545/) (2026)

## Abstract (arXiv)

The main challenge that sets transfer learning apart from traditional supervised learning is the distribution shift, reflected as the shift between the source and target models and that between the marginal covariate distributions. In this work, we tackle model shifts in the presence of covariate shifts in the high-dimensional regression setting. Specifically, we propose a two-step method with a novel fused-regularizer that effectively leverages samples from source tasks to improve the learning performance on a target task with limited samples. Nonasymptotic bound is provided for the estimation error of the target model, showing the robustness of the proposed method to covariate shifts. We further establish conditions under which the estimator is minimax-optimal. Additionally, we extend the method to a distributed setting, allowing for a pretraining-finetuning strategy, requiring just one round of communication while retaining the estimation rate of the centralized version. Numerical tests validate our theory, highlighting the method's robustness to covariate shifts.