# Generalization error of min-norm interpolators in transfer learning

arXiv: https://arxiv.org/abs/2406.13944 · HTML: https://mlsanity.com/paper/2406.13944/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jun 2024.

## Built on by (most cited first, 16 of 16)

- [High-dimensional Analysis of Knowledge Distillation: Weak-to-Strong Generalization and Scaling Laws](https://mlsanity.com/paper/2410.18837/) (2024)
- [Spurious Correlations in High Dimensional Regression: The Roles of Regularization, Simplicity Bias and Over-Parameterization](https://mlsanity.com/paper/2502.01347/) (2025)
- [Preventing Model Collapse Under Overparametrization: Optimal Mixing Ratios for Interpolation Learning and Ridge Regression](https://mlsanity.com/paper/2509.22341/) (2025)
- [High-dimensional Analysis of Synthetic Data Selection](https://mlsanity.com/paper/2510.08123/) (2025)
- [Safety vs. Performance: How Multi-Objective Learning Reduces Barriers to Market Entry](https://mlsanity.com/paper/2409.03734/) (2024)
- [TL-PCA: Transfer Learning of Principal Component Analysis](https://mlsanity.com/paper/2410.10805/) (2024)
- [Transfer Learning for Benign Overfitting in High-Dimensional Linear Regression](https://mlsanity.com/paper/2510.15337/) (2025)
- [Optimal and Provable Calibration in High-Dimensional Binary Classification: Angular Calibration and Platt Scaling](https://mlsanity.com/paper/2502.15131/) (2025)
- [Transfer learning via Regularized Linear Discriminant Analysis](https://mlsanity.com/paper/2501.02411/) (2025)
- [Optimal Regularization for Performative Learning](https://mlsanity.com/paper/2510.12249/) (2025)
- [Optimal Representation Size: High-Dimensional Analysis of Pretraining and Linear Probing](https://mlsanity.com/paper/2605.20105/) (2026)
- [Multi-Environment GLAMP: Approximate Message Passing for Transfer Learning with Applications to Lasso-based Estimators](https://mlsanity.com/paper/2505.22594/) (2025)
- [Prediction-Only Distillation in Linear and Logistic Regression](https://mlsanity.com/paper/2607.15450/) (2026)
- [The Zero Pattern of a Design Matrix Drives Multiple Descent in Over-parameterized Regression](https://mlsanity.com/paper/2607.24041/) (2026)
- [When do data mixtures improve scaling laws? Insights from high-dimensional regression](https://mlsanity.com/paper/2609.38011/) (2026)
- [High-Dimensional Asymptotics and Dataset Selection for Private Transfer Learning](https://mlsanity.com/paper/2610.02578/) (2026)

## Abstract (arXiv)

This paper establishes the generalization error of pooled min-ℓ2-norm interpolation in transfer learning, where data from diverse distributions are available. Min-norm interpolators arise naturally as implicit regularized limits of modern machine learning algorithms. Prior work has characterized their out-of-distribution risk when samples from the test distribution are unavailable during training. In many applications, however, limited test samples may be available at training time, yet properties of min-norm interpolation in this regime remain poorly understood. We address this gap by characterizing the bias and variance of pooled min-ℓ2-norm interpolation under both covariate shift and model shift. Our results yield several important implications. In certain cases under model shift, we show that adding data always hurts when the signal-to-noise ratio (SNR) is low. At higher SNR levels, transfer learning is beneficial provided the shift-to-signal ratio falls below a threshold that we characterize explicitly. Under covariate shift, we find that when the source sample size is small relative to the dimension, greater heterogeneity between domains reduces risk, and vice versa. While our model shift results are initially established for Gaussian designs, we extend them to more general designs through a universality argument. To illustrate the broader applicability of our technical tools beyond interpolation learning, we characterize the risk of a bias-corrected estimator that uses the pooled interpolator as an initialization and corrects the resulting bias with target data. On the technical side, we develop a novel anisotropic local law and a Lindeberg-swapping argument, yielding tools that may be of independent interest in random matrix theory and universality analysis. Finally, we supplement our theory with simulations demonstrating the finite-sample efficacy of our results.