Fixed-Gaussian Spectral Algorithms: Minimax Optimal Rates for Misspecified Learning and Transfer
Not on the map. Why?
Abstract (arXiv)
The principal objective of this work is twofold within nonparametric regression settings: (1) to establish the minimax optimal convergence rates for fixed-bandwidth Gaussian kernel spectral algorithms when the true regression function resides in a Sobolev space, and (2) to apply Gaussian spectral algorithms for achieving robust and adaptive transfer learning under concept shift. While minimax optimality of misspecified spectral algorithms has been established, existing guarantees are typically restricted to the non-saturation regime. We demonstrate that the infinite smoothness of fixed-bandwidth Gaussian kernels provides universal robustness to model misspecification by showing that this kernel choice enables any spectral algorithm to attain minimax optimal rates, provided the regularization parameter decays exponentially. This result effectively decouples optimality from the algorithm's inherent qualification. Building on this, we then advocate Gaussian spectral algorithms as powerful components in a learning framework for robust and adaptive transfer. Specifically, we derive the adaptive convergence rate of the excess risk for this framework and show that the rates are optimal up to logarithmic factors. Our results also reveal the impact of the magnitude of the concept shift and the sample size on the generalization error.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 45 of its 85 references that are in the map. First posted on arXiv 18 Jan 2025.
Builds on
45 of 85 references.
References · 45
- 2014How transferable are features in deep neural networks?cited by 1,216
- 2019A Comprehensive Survey on Transfer Learningcited by 1,154
- 2018Gaussian Processes and Kernel Methods: A Review on Connections and Equivalencescited by 188
- 2017Sobolev Norm Learning Rates for Regularized Least-Squares Algorithmcited by 113
- 2020Transfer Learning for High-dimensional Linear Regression: Prediction, Estimation, and Minimax Optimalitycited by 89
- 2018Statistical Optimality of Stochastic Gradient Descent on Hard Learning Problems through Multiple Passescited by 69
- 2021Transfer Learning under High-dimensional Generalized Linear Modelscited by 69
- 2016Optimal Rates For Regularization Of Statistical Inverse Learning Problemscited by 68
- 2018Marginal Singularity, and the Benefits of Labels in Covariate-Shiftcited by 65
- 2013Divide and Conquer Kernel Ridge Regression: A Distributed Algorithm with Minimax Optimal Ratescited by 64
- 2019Transfer Learning for Nonparametric Classification: Minimax Rate and Adaptive Classifiercited by 64
- 2018Optimal Rates for Spectral Algorithms with Least-Squares Regression over Hilbert Spacescited by 59
33 more
- 2022Optimally tackling covariate shift in RKHS-based nonparametric regressioncited by 53
- 2016Distributed learning with regularized least squarescited by 42
- 2021Near-Optimal Linear Regression under Distribution Shiftcited by 40
- 2020Transfer Learning in Large-scale Gaussian Graphical Models with False Discovery Rate Controlcited by 38
- 2023On the Optimality of Misspecified Spectral Algorithmscited by 28
- 2016Hypothesis Transfer Learning via Transformation Functionscited by 26
- 2018Optimal Convergence for Distributed Learning with Stochastic Gradient Methods and Spectral Algorithmscited by 22
- 2018A Generalized Neyman-Pearson Criterion for Optimal Domain Adaptationcited by 22
- 2019Beating SGD Saturation with Tail-Averaging and Minibatchingcited by 21
- 2020Minimax optimal approaches to the label shift problem in non-parametric settingscited by 20
- 2023On the Optimality of Misspecified Kernel Ridge Regressioncited by 20
- 2022Spectral Regularized Kernel Two-Sample Testscited by 19
- 2024Transfer Learning for Nonparametric Regression: Non-asymptotic Minimax Analysis and Adaptive Procedurecited by 19
- 2023Pseudo-Labeling for Kernel Ridge Regression under Covariate Shiftcited by 17
- 2014Fast Rates by Transferring from Auxiliary Hypothesescited by 16
- 2021A linear adjustment based approach to posterior drift in transfer learningcited by 16
- 2022Doubly Robust Augmented Model Accuracy Transfer Inference with High Dimensional Featurescited by 12
- 2022Semi-supervised Triply Robust Inductive Transfer Learningcited by 12
- 2024On the Saturation Effect of Kernel Ridge Regressioncited by 12
- 2024Optimal Rates for Vector-Valued Spectral Regularization Learning Algorithmscited by 12
- 2016Optimal rates for the regularized learning algorithms under general source conditioncited by 11
- 2020Augmented Transfer Regression Learning with Semi-non-parametric Nuisance Modelscited by 11
- 2022A Class of Geometric Structures in Transfer Learning: Minimax Bounds and Optimalitycited by 9
- 2024Optimal Rates of Kernel Ridge Regression under Source Condition in Large Dimensionscited by 9
- 2024Generalization Error Curves for Analytic Spectral Algorithms under Power-law Decaycited by 8
- 2024Smoothness Adaptive Hypothesis Transfer Learningcited by 8
- 2023Spectral Regularized Kernel Goodness-of-Fit Testscited by 6
- 2020Inverse learning in Hilbert scalescited by 5
- 2024Domain Adaptation Targeting Heterogeneous and Imbalanced Subgroupscited by 5
- 2020A General Class of Transfer Learning Regression without Implementation Costcited by 3
- 2022Transfer learning with affine model transformationcited by 3
- 2024On the Approximation of Kernel functionscited by 3
- 2022On Hypothesis Transfer Learning of Functional Linear Modelscited by 2
Built on by
Citing papers per year: 2026: 2.
2 papers.