# Minimum-Norm Interpolation Under Covariate Shift

arXiv: https://arxiv.org/abs/2404.00522 · HTML: https://mlsanity.com/paper/2404.00522/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Mar 2024.

## Built on by (most cited first, 14 of 14)

- [High-dimensional Analysis of Knowledge Distillation: Weak-to-Strong Generalization and Scaling Laws](https://mlsanity.com/paper/2410.18837/) (2024)
- [Spurious Correlations in High Dimensional Regression: The Roles of Regularization, Simplicity Bias and Over-Parameterization](https://mlsanity.com/paper/2502.01347/) (2025)
- [Preventing Model Collapse Under Overparametrization: Optimal Mixing Ratios for Interpolation Learning and Ridge Regression](https://mlsanity.com/paper/2509.22341/) (2025)
- [High-dimensional Analysis of Synthetic Data Selection](https://mlsanity.com/paper/2510.08123/) (2025)
- [Safety vs. Performance: How Multi-Objective Learning Reduces Barriers to Market Entry](https://mlsanity.com/paper/2409.03734/) (2024)
- [Learning where to learn: Training data distribution optimization for scientific machine learning](https://mlsanity.com/paper/2505.21626/) (2025)
- [Transfer Learning for Benign Overfitting in High-Dimensional Linear Regression](https://mlsanity.com/paper/2510.15337/) (2025)
- [Optimal Regularization for Performative Learning](https://mlsanity.com/paper/2510.12249/) (2025)
- [Beyond the Training Distribution: Evaluating Predictions Under Distribution Shift and Selection Bias](https://mlsanity.com/paper/2606.14506/) (2026)
- [Lassoed Forests: Random Forests with Adaptive Lasso Post-selection](https://mlsanity.com/paper/2511.06698/) (2025)
- [Supervised Fine-Tuning vs. In-Context Learning: An Equilibrium Analysis of LLM Personalization under Congestion](https://mlsanity.com/paper/2607.14371/) (2026)
- [Prediction-Only Distillation in Linear and Logistic Regression](https://mlsanity.com/paper/2607.15450/) (2026)
- [Benign Overfitting for General Norms and Distributions](https://mlsanity.com/paper/2609.33675/) (2026)
- [High-Dimensional Asymptotics and Dataset Selection for Private Transfer Learning](https://mlsanity.com/paper/2610.02578/) (2026)

## Abstract (arXiv)

Transfer learning is a critical part of real-world machine learning deployments and has been extensively studied in experimental works with overparameterized neural networks. However, even in the simplest setting of linear regression a notable gap still exists in the theoretical understanding of transfer learning. In-distribution research on high-dimensional linear regression has led to the identification of a phenomenon known as textitbenign overfitting, in which linear interpolators overfit to noisy training labels and yet still generalize well. This behavior occurs under specific conditions on the source covariance matrix and input data dimension. Therefore, it is natural to wonder how such high-dimensional linear models behave under transfer learning. We prove the first non-asymptotic excess risk bounds for benignly-overfit linear interpolators in the transfer learning setting. From our analysis, we propose a taxonomy of textitbeneficial and textitmalignant covariate shifts based on the degree of overparameterization. We follow our analysis with empirical studies that show these beneficial and malignant covariate shifts for linear interpolators on real image data, and for fully-connected neural networks in settings where the input data dimension is larger than the training sample size.