# Transfer Learning in Information Criteria-based Feature Selection

arXiv: https://arxiv.org/abs/2107.02847 · HTML: https://mlsanity.com/paper/2107.02847/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 59 references that are in the map.
- First posted on arXiv 6 Jul 2021.

## Builds on (most cited first, 2 of 2)

- [The ALAMO approach to machine learning](https://mlsanity.com/paper/1705.10918/) (2017)
- [No penalty no tears: Least squares in high-dimensional linear models](https://mlsanity.com/paper/1506.02222/) (2015)

## Built on by (most cited first, 1 of 1)

- [A Scalable and Efficient Iterative Method for Copying Machine Learning Classifiers](https://mlsanity.com/paper/2302.02667/) (2023)

## Abstract (arXiv)

This paper investigates the effectiveness of transfer learning based on Mallows' Cp. We propose a procedure that combines transfer learning with Mallows' Cp (TLCp) and prove that it outperforms the conventional Mallows' Cp criterion in terms of accuracy and stability. Our theoretical results indicate that, for any sample size in the target domain, the proposed TLCp estimator performs better than the Cp estimator by the mean squared error (MSE) metric in the case of orthogonal predictors, provided that i) the dissimilarity between the tasks from source domain and target domain is small, and ii) the procedure parameters (complexity penalties) are tuned according to certain explicit rules. Moreover, we show that our transfer learning framework can be extended to other feature selection criteria, such as the Bayesian information criterion. By analyzing the solution of the orthogonalized Cp, we identify an estimator that asymptotically approximates the solution of the Cp criterion in the case of non-orthogonal predictors. Similar results are obtained for the non-orthogonal TLCp. Finally, simulation studies and applications with real data demonstrate the usefulness of the TLCp scheme.