# Transfer Learning for Linear Regression: a Statistical Test of Gain

arXiv: https://arxiv.org/abs/2102.09504 · HTML: https://mlsanity.com/paper/2102.09504/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 18 references that are in the map.
- First posted on arXiv 18 Feb 2021.

## Builds on (most cited first, 4 of 4)

- [Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning](https://mlsanity.com/paper/1602.03409/) (2016)
- [Transfer learning for time series classification](https://mlsanity.com/paper/1811.01533/) (2018)
- [Double Double Descent: On Generalization Errors in Transfer Learning between Linear Regression Tasks](https://mlsanity.com/paper/2006.07002/) (2020)
- [Data enriched linear regression](https://mlsanity.com/paper/1304.1837/) (2013)

## Built on by (most cited first, 10 of 10)

- [A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learning](https://mlsanity.com/paper/2109.02355/) (2021)
- [The Common Intuition to Transfer Learning Can Win or Lose: Case Studies for Linear Regression](https://mlsanity.com/paper/2103.05621/) (2021)
- [Transfer Learning with Kernel Methods](https://mlsanity.com/paper/2211.00227/) (2022)
- [Hierarchical transfer learning with applications for electricity load forecasting](https://mlsanity.com/paper/2111.08512/) (2021)
- [TL-PCA: Transfer Learning of Principal Component Analysis](https://mlsanity.com/paper/2410.10805/) (2024)
- [Transfer-Learning Across Datasets with Different Input Dimensions: An Algorithm and Analysis for the Linear Regression Case](https://mlsanity.com/paper/2202.05069/) (2022)
- [ISTRBoost: Importance Sampling Transfer Regression using Boosting](https://mlsanity.com/paper/2204.12044/) (2022)
- [When to Transfer: Adaptive Source Selection for Positive Transfer in Linear Models](https://mlsanity.com/paper/2510.16986/) (2025)
- [Transfer Learning of Linear Regression with Multiple Pretrained Models: Benefiting from More Pretrained Models via Overparameterization Debiasing](https://mlsanity.com/paper/2602.16531/) (2026)
- [Tunable Domain Adaptation Using Unfolding](https://mlsanity.com/paper/2603.26931/) (2026)

## Abstract (arXiv)

Transfer learning, also referred as knowledge transfer, aims at reusing knowledge from a source dataset to a similar target one. While many empirical studies illustrate the benefits of transfer learning, few theoretical results are established especially for regression problems. In this paper a theoretical framework for the problem of parameter transfer for the linear model is proposed. It is shown that the quality of transfer for a new input vector x depends on its representation in an eigenbasis involving the parameters of the problem. Furthermore a statistical test is constructed to predict whether a fine-tuned model has a lower prediction quadratic risk than the base target model for an unobserved sample. Efficiency of the test is illustrated on synthetic data as well as real electricity consumption data.