# AdaTrans: Feature-wise and Sample-wise Adaptive Transfer Learning for High-dimensional Regression

arXiv: https://arxiv.org/abs/2403.13565 · HTML: https://mlsanity.com/paper/2403.13565/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 27 references that are in the map.
- First posted on arXiv 20 Mar 2024.

## Builds on (most cited first, 13 of 13)

- [Ditto: Fair and Robust Federated Learning Through Personalization](https://mlsanity.com/paper/2012.04221/) (2020)
- [Rethinking Importance Weighting for Deep Learning under Distribution Shift](https://mlsanity.com/paper/2006.04662/) (2020)
- [Transfer Learning for High-dimensional Linear Regression: Prediction, Estimation, and Minimax Optimality](https://mlsanity.com/paper/2006.10593/) (2020)
- [Transfer Learning under High-dimensional Generalized Linear Models](https://mlsanity.com/paper/2105.14328/) (2021)
- [Transfer Learning in Large-scale Gaussian Graphical Models with False Discovery Rate Control](https://mlsanity.com/paper/2010.11037/) (2020)
- [Domain Aggregation Networks for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1909.05352/) (2019)
- [Personalized Federated Learning: A Unified Framework and Universal Optimization Techniques](https://mlsanity.com/paper/2102.09743/) (2021)
- [Generalization Bounds for Domain Adaptation](https://mlsanity.com/paper/1304.1574/) (2013)
- [Robust Learning from Untrusted Sources](https://mlsanity.com/paper/1901.10310/) (2019)
- [A One-step Approach to Covariate Shift Adaptation](https://mlsanity.com/paper/2007.04043/) (2020)
- [Rethinking Importance Weighting for Transfer Learning](https://mlsanity.com/paper/2112.10157/) (2021)
- [Distributed Personalized Empirical Risk Minimization](https://mlsanity.com/paper/2310.17761/) (2023)
- [Unified Transfer Learning Models in High-Dimensional Linear Regression](https://mlsanity.com/paper/2307.00238/) (2023)

## Built on by (most cited first, 3 of 3)

- [Understanding the Statistical Accuracy-Communication Trade-off in Personalized Federated Learning with Minimax Guarantees](https://mlsanity.com/paper/2410.08934/) (2024)
- [Multi-task Learning for Heterogeneous Multi-source Block-Wise Missing Data](https://mlsanity.com/paper/2505.24413/) (2025)
- [Bytes to Threats: Byte-Sequence Transformer For Malware Detection](https://mlsanity.com/paper/2610.18787/) (2026)

## Abstract (arXiv)

We consider the transfer learning problem in the high dimensional linear regression setting, where the feature dimension is larger than the sample size. To learn transferable information, which may vary across features or the source samples, we propose an adaptive transfer learning method that can detect and aggregate the feature-wise (F-AdaTrans) or sample-wise (S-AdaTrans) transferable structures. We achieve this by employing a fused-penalty, coupled with weights that can adapt according to the transferable structure. To choose the weight, we propose a theoretically informed, data-driven procedure, enabling F-AdaTrans to selectively fuse the transferable signals with the target while filtering out non-transferable signals, and S-AdaTrans to obtain the optimal combination of information transferred from each source sample. We show that, with appropriately chosen weights, F-AdaTrans achieves a convergence rate close to that of an oracle estimator with a known transferable structure, and S-AdaTrans recovers existing near-minimax optimal rates as a special case. The effectiveness of the proposed method is validated using both simulation and real data, demonstrating favorable performance compared to the existing methods.