mlsanityData to 7 Oct 2026

Smoothness Adaptive Hypothesis Transfer Learning

arXiv 2402.14966 ↗First posted 22 Feb 2024stat.ML cs.LG stat.MEcited by 8

Not on the map. Why?

Abstract (arXiv)

Many existing two-phase kernel-based hypothesis transfer learning algorithms employ the same kernel regularization across phases and rely on the known smoothness of functions to obtain optimality. Therefore, they fail to adapt to the varying and unknown smoothness between the target/source and their offset in practice. In this paper, we address these problems by proposing Smoothness Adaptive Transfer Learning (SATL), a two-phase kernel ridge regression(KRR)-based algorithm. We first prove that employing the misspecified fixed bandwidth Gaussian kernel in target-only KRR learning can achieve minimax optimality and derive an adaptive procedure to the unknown Sobolev smoothness. Leveraging these results, SATL employs Gaussian kernels in both phases so that the estimators can adapt to the unknown smoothness of the target/source and their offset function. We derive the minimax lower bound of the learning problem in excess risk and show that SATL enjoys a matching upper bound up to a logarithmic factor. The minimax convergence rate sheds light on the factors influencing transfer dynamics and demonstrates the superiority of SATL compared to non-transfer learning settings. While our main objective is a theoretical analysis, we also conduct several experiments to confirm our results.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 18 of its 46 references that are in the map. First posted on arXiv 22 Feb 2024.

Builds on

18 of 46 references.

References · 18

  1. 2020Few-Shot Learning via Learning the Representation, Provablycited by 213
  2. 2018Gaussian Processes and Kernel Methods: A Review on Connections and Equivalencescited by 188
  3. 2020On the Theory of Transfer Learning: The Importance of Task Diversitycited by 186
  4. 2017Sobolev Norm Learning Rates for Regularized Least-Squares Algorithmcited by 113
  5. 2020Transfer Learning for High-dimensional Linear Regression: Prediction, Estimation, and Minimax Optimalitycited by 89
  6. 2021Transfer Learning under High-dimensional Generalized Linear Modelscited by 69
  7. 2022Optimally tackling covariate shift in RKHS-based nonparametric regressioncited by 53
  8. 2022Adaptive and Robust Multi-Task Learningcited by 33
  9. 2023Learning from Similar Linear Representations: Adaptivity, Minimaxity, and Robustnesscited by 27
  10. 2016Hypothesis Transfer Learning via Transformation Functionscited by 26
  11. 2019On the Optimality of Gaussian Kernel Based Nonparametric Tests against Smooth Alternativescited by 21
  12. 2021Representation Learning Beyond Linear Prediction Functionscited by 21
6 more
  1. 2023On the Optimality of Misspecified Kernel Ridge Regressioncited by 20
  2. 2024Transfer Learning for Nonparametric Regression: Non-asymptotic Minimax Analysis and Adaptive Procedurecited by 19
  3. 2023Pseudo-Labeling for Kernel Ridge Regression under Covariate Shiftcited by 17
  4. 2014Fast Rates by Transferring from Auxiliary Hypothesescited by 16
  5. 2022A Class of Geometric Structures in Transfer Learning: Minimax Bounds and Optimalitycited by 9
  6. 2023Hypothesis Transfer Learning with Surrogate Classification Losses: Generalization Bounds through Algorithmic Stabilitycited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 2.

8 papers.

  1. 2024TransFusion: Covariate-Shift Robust Transfer Learning for High-Dimensional Regressioncited by 11
  2. 2025Fixed-Gaussian Spectral Algorithms: Minimax Optimal Rates for Misspecified Learning and Transfercited by 2
  3. 2025Wasserstein Transfer Learningcited by 2
  4. 2025Transfer Learning for Benign Overfitting in High-Dimensional Linear Regressioncited by 2
  5. 2025Sharp Gaussian approximations for Decentralized Federated Learningcited by 1
  6. 2026Transfer Learning Through Conditional Quantile Matchingcited by 1
  7. 2025Transfer Learning for High-dimensional Reduced Rank Time Series Modelscited by 0
  8. 2026Transfer Learning in Nonparametric Regression with Deep ReLU Networkscited by 0