mlsanityData to 7 Oct 2026

TransFusion: Covariate-Shift Robust Transfer Learning for High-Dimensional Regression

arXiv 2404.01153 ↗First posted 1 Apr 2024stat.ML cs.DC cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

The main challenge that sets transfer learning apart from traditional supervised learning is the distribution shift, reflected as the shift between the source and target models and that between the marginal covariate distributions. In this work, we tackle model shifts in the presence of covariate shifts in the high-dimensional regression setting. Specifically, we propose a two-step method with a novel fused-regularizer that effectively leverages samples from source tasks to improve the learning performance on a target task with limited samples. Nonasymptotic bound is provided for the estimation error of the target model, showing the robustness of the proposed method to covariate shifts. We further establish conditions under which the estimator is minimax-optimal. Additionally, we extend the method to a distributed setting, allowing for a pretraining-finetuning strategy, requiring just one round of communication while retaining the estimation rate of the centralized version. Numerical tests validate our theory, highlighting the method's robustness to covariate shifts.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 14 of its 44 references that are in the map. First posted on arXiv 1 Apr 2024.

Builds on

14 of 44 references.

References · 14

  1. 2020Few-Shot Learning via Learning the Representation, Provablycited by 213
  2. 2020On the Theory of Transfer Learning: The Importance of Task Diversitycited by 186
  3. 2019MNIST-C: A Robustness Benchmark for Computer Visioncited by 137
  4. 2020A survey on domain adaptation theory: learning bounds and theoretical guaranteescited by 109
  5. 2020Transfer Learning for High-dimensional Linear Regression: Prediction, Estimation, and Minimax Optimalitycited by 89
  6. 2013Confidence Intervals and Hypothesis Testing for High-Dimensional Regressioncited by 78
  7. 2021Transfer Learning under High-dimensional Generalized Linear Modelscited by 69
  8. 2021Near-Optimal Linear Regression under Distribution Shiftcited by 40
  9. 2023Learning from Similar Linear Representations: Adaptivity, Minimaxity, and Robustnesscited by 27
  10. 2016Hypothesis Transfer Learning via Transformation Functionscited by 26
  11. 2016A highly efficient semismooth Newton augmented Lagrangian method for solving Lasso problemscited by 14
  12. 2024Smoothness Adaptive Hypothesis Transfer Learningcited by 8
2 more
  1. 2023Optimal Multitask Linear Regression and Contextual Bandits under Sparse Heterogeneitycited by 7
  2. 2023Unified Transfer Learning Models in High-Dimensional Linear Regressioncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 7; 2026: 2.

11 papers.

  1. 2024Trans-Glasso: A Transfer Learning Approach to Precision Matrix Estimationcited by 2
  2. 2025Transfer Learning for Benign Overfitting in High-Dimensional Linear Regressioncited by 2
  3. 2024Understanding the Statistical Accuracy-Communication Trade-off in Personalized Federated Learning with Minimax Guaranteescited by 1
  4. 2025Multi-task Learning for Heterogeneous Multi-source Block-Wise Missing Datacited by 1
  5. 2026Minimum Wasserstein distance estimator under covariate shift: closed-form, super-efficiency and irregularitycited by 1
  6. 2025Post-Transfer Learning Statistical Inference in High-Dimensional Regressioncited by 0
  7. 2025Transfer Learning for Matrix Completioncited by 0
  8. 2025Representation-Aware Distributionally Robust Optimization: A Knowledge Transfer Frameworkcited by 0
  9. 2025Transfer learning under latent space modelcited by 0
  10. 2025Transfer Learning with Distance Covariance for Random Forest: Error Bounds and an EHR Applicationcited by 0
  11. 2026Sample Complexity of Transfer Learning: An Optimal Transport Approachcited by 0