# Minimax Lower Bounds for Transfer Learning with Linear and One-hidden Layer Neural Networks

arXiv: https://arxiv.org/abs/2006.10581 · HTML: https://mlsanity.com/paper/2006.10581/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jun 2020.

## Built on by (most cited first, 30 of 30)

- [Near-Optimal Linear Regression under Distribution Shift](https://mlsanity.com/paper/2106.12108/) (2021)
- [Adaptive and Robust Multi-Task Learning](https://mlsanity.com/paper/2202.05250/) (2022)
- [A new similarity measure for covariate shift with applications to nonparametric regression](https://mlsanity.com/paper/2202.02837/) (2022)
- [Weighted Training for Cross-Task Learning](https://mlsanity.com/paper/2105.14095/) (2021)
- [First Steps Toward Understanding the Extrapolation of Nonlinear Models to Unseen Domains](https://mlsanity.com/paper/2211.11719/) (2022)
- [Explaining the physics of transfer learning a data-driven subgrid-scale closure to a different turbulent flow](https://mlsanity.com/paper/2206.03198/) (2022)
- [Pseudo-Labeling for Kernel Ridge Regression under Covariate Shift](https://mlsanity.com/paper/2302.10160/) (2023)
- [Maximum Likelihood Estimation is All You Need for Well-Specified Covariate Shift](https://mlsanity.com/paper/2311.15961/) (2023)
- [Characterizing and Understanding the Generalization Error of Transfer Learning with Gibbs Algorithm](https://mlsanity.com/paper/2111.01635/) (2021)
- [On the Statistical Benefits of Curriculum Learning](https://mlsanity.com/paper/2111.07126/) (2021)
- [A Class of Geometric Structures in Transfer Learning: Minimax Bounds and Optimality](https://mlsanity.com/paper/2202.11685/) (2022)
- [Limits of Model Selection under Transfer Learning](https://mlsanity.com/paper/2305.00152/) (2023)
- [Analysing Multi-Task Regression via Random Matrix Theory with Application to Time Series Forecasting](https://mlsanity.com/paper/2406.10327/) (2024)
- [Double Debiased Covariate Shift Adaptation Robust to Density-Ratio Estimation](https://mlsanity.com/paper/2310.16638/) (2023)
- [Transfer Learning Beyond Bounded Density Ratios](https://mlsanity.com/paper/2403.11963/) (2024)
- [Risk of Transfer Learning and its Applications in Finance](https://mlsanity.com/paper/2311.03283/) (2023)
- [Gap Minimization for Knowledge Sharing and Transfer](https://mlsanity.com/paper/2201.11231/) (2022)
- [Help Me Help You: The Aggregate Value of Source and Target Data in Transfer Learning](https://mlsanity.com/paper/2408.16189/) (2024)
- [Learning Neural Networks with Distribution Shift: Efficiently Certifiable Guarantees](https://mlsanity.com/paper/2502.16021/) (2025)
- [Tight Rates in Supervised Outlier Transfer Learning](https://mlsanity.com/paper/2310.04686/) (2023)
- [Feasibility of Transfer Learning: A Mathematical Framework](https://mlsanity.com/paper/2305.12985/) (2023)
- [Neyman-Pearson Classification under Both Null and Alternative Distributions Shift](https://mlsanity.com/paper/2511.06641/) (2025)
- [Statistical curriculum learning: An elimination algorithm achieving an oracle risk](https://mlsanity.com/paper/2402.13366/) (2024)
- [Quantum Curriculum Learning](https://mlsanity.com/paper/2407.02419/) (2024)
- [Understanding Transfer Learning via Mean-field Analysis](https://mlsanity.com/paper/2410.17128/) (2024)
- [Mixed-Sample SGD: an End-to-end Analysis of Supervised Transfer Learning](https://mlsanity.com/paper/2507.04194/) (2025)
- [Transfer-Learning Across Datasets with Different Input Dimensions: An Algorithm and Analysis for the Linear Regression Case](https://mlsanity.com/paper/2202.05069/) (2022)
- [Learning When the Concept Shifts: Confounding, Invariance, and Dimension Reduction](https://mlsanity.com/paper/2406.15904/) (2024)
- [Expectation Error Bounds for Transfer Learning in Linear Regression and Linear Neural Networks](https://mlsanity.com/paper/2603.28739/) (2026)
- [Structuring Relations Among Learning Paradigms via Protocol--Objective--Resource Reductions](https://mlsanity.com/paper/2609.32766/) (2026)

## Abstract (arXiv)

Transfer learning has emerged as a powerful technique for improving the performance of machine learning models on new domains where labeled training data may be scarce. In this approach a model trained for a source task, where plenty of labeled training data is available, is used as a starting point for training a model on a related target task with only few labeled training data. Despite recent empirical success of transfer learning approaches, the benefits and fundamental limits of transfer learning are poorly understood. In this paper we develop a statistical minimax framework to characterize the fundamental limits of transfer learning in the context of regression with linear and one-hidden layer neural network models. Specifically, we derive a lower-bound for the target generalization error achievable by any algorithm as a function of the number of labeled source and target data as well as appropriate notions of similarity between the source and target tasks. Our lower bound provides new insights into the benefits and limitations of transfer learning. We further corroborate our theoretical finding with various experiments.