# Smoothness Adaptive Hypothesis Transfer Learning

arXiv: https://arxiv.org/abs/2402.14966 · HTML: https://mlsanity.com/paper/2402.14966/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 46 references that are in the map.
- First posted on arXiv 22 Feb 2024.

## Builds on (most cited first, 18 of 18)

- [Few-Shot Learning via Learning the Representation, Provably](https://mlsanity.com/paper/2002.09434/) (2020)
- [Gaussian Processes and Kernel Methods: A Review on Connections and Equivalences](https://mlsanity.com/paper/1807.02582/) (2018)
- [On the Theory of Transfer Learning: The Importance of Task Diversity](https://mlsanity.com/paper/2006.11650/) (2020)
- [Sobolev Norm Learning Rates for Regularized Least-Squares Algorithm](https://mlsanity.com/paper/1702.07254/) (2017)
- [Transfer Learning for High-dimensional Linear Regression: Prediction, Estimation, and Minimax Optimality](https://mlsanity.com/paper/2006.10593/) (2020)
- [Transfer Learning under High-dimensional Generalized Linear Models](https://mlsanity.com/paper/2105.14328/) (2021)
- [Optimally tackling covariate shift in RKHS-based nonparametric regression](https://mlsanity.com/paper/2205.02986/) (2022)
- [Adaptive and Robust Multi-Task Learning](https://mlsanity.com/paper/2202.05250/) (2022)
- [Learning from Similar Linear Representations: Adaptivity, Minimaxity, and Robustness](https://mlsanity.com/paper/2303.17765/) (2023)
- [Hypothesis Transfer Learning via Transformation Functions](https://mlsanity.com/paper/1612.01020/) (2016)
- [Transfer Learning for Nonparametric Regression: Non-asymptotic Minimax Analysis and Adaptive Procedure](https://mlsanity.com/paper/2401.12272/) (2024)
- [On the Optimality of Gaussian Kernel Based Nonparametric Tests against Smooth Alternatives](https://mlsanity.com/paper/1909.03302/) (2019)
- [Representation Learning Beyond Linear Prediction Functions](https://mlsanity.com/paper/2105.14989/) (2021)
- [On the Optimality of Misspecified Kernel Ridge Regression](https://mlsanity.com/paper/2305.07241/) (2023)
- [Pseudo-Labeling for Kernel Ridge Regression under Covariate Shift](https://mlsanity.com/paper/2302.10160/) (2023)
- [Fast Rates by Transferring from Auxiliary Hypotheses](https://mlsanity.com/paper/1412.1619/) (2014)
- [A Class of Geometric Structures in Transfer Learning: Minimax Bounds and Optimality](https://mlsanity.com/paper/2202.11685/) (2022)
- [Hypothesis Transfer Learning with Surrogate Classification Losses: Generalization Bounds through Algorithmic Stability](https://mlsanity.com/paper/2305.19694/) (2023)

## Built on by (most cited first, 8 of 8)

- [TransFusion: Covariate-Shift Robust Transfer Learning for High-Dimensional Regression](https://mlsanity.com/paper/2404.01153/) (2024)
- [Transfer Learning for Benign Overfitting in High-Dimensional Linear Regression](https://mlsanity.com/paper/2510.15337/) (2025)
- [Fixed-Gaussian Spectral Algorithms: Minimax Optimal Rates for Misspecified Learning and Transfer](https://mlsanity.com/paper/2501.10870/) (2025)
- [Wasserstein Transfer Learning](https://mlsanity.com/paper/2505.17404/) (2025)
- [Sharp Gaussian approximations for Decentralized Federated Learning](https://mlsanity.com/paper/2505.08125/) (2025)
- [Transfer Learning Through Conditional Quantile Matching](https://mlsanity.com/paper/2602.02358/) (2026)
- [Transfer Learning for High-dimensional Reduced Rank Time Series Models](https://mlsanity.com/paper/2504.15691/) (2025)
- [Transfer Learning in Nonparametric Regression with Deep ReLU Networks](https://mlsanity.com/paper/2608.20255/) (2026)

## Abstract (arXiv)

Many existing two-phase kernel-based hypothesis transfer learning algorithms employ the same kernel regularization across phases and rely on the known smoothness of functions to obtain optimality. Therefore, they fail to adapt to the varying and unknown smoothness between the target/source and their offset in practice. In this paper, we address these problems by proposing Smoothness Adaptive Transfer Learning (SATL), a two-phase kernel ridge regression(KRR)-based algorithm. We first prove that employing the misspecified fixed bandwidth Gaussian kernel in target-only KRR learning can achieve minimax optimality and derive an adaptive procedure to the unknown Sobolev smoothness. Leveraging these results, SATL employs Gaussian kernels in both phases so that the estimators can adapt to the unknown smoothness of the target/source and their offset function. We derive the minimax lower bound of the learning problem in excess risk and show that SATL enjoys a matching upper bound up to a logarithmic factor. The minimax convergence rate sheds light on the factors influencing transfer dynamics and demonstrates the superiority of SATL compared to non-transfer learning settings. While our main objective is a theoretical analysis, we also conduct several experiments to confirm our results.