mlsanityData to 7 Oct 2026

A Theoretical Analysis of the Test Error of Finite-Rank Kernel Ridge Regression

arXiv 2310.00987 ↗First posted 2 Oct 2023cs.LG stat.MLcited by 8

Not on the map. Why?

Abstract (arXiv)

Existing statistical learning guarantees for general kernel regressors often yield loose bounds when used with finite-rank kernels. Yet, finite-rank kernels naturally appear in several machine learning problems, e.g. when fine-tuning a pre-trained deep neural network's last layer to adapt it to a novel task when performing transfer learning. We address this gap for finite-rank kernel ridge regression (KRR) by deriving sharp non-asymptotic upper and lower bounds for the KRR test error of any finite-rank KRR. Our bounds are tighter than previously derived bounds on finite-rank KRR, and unlike comparable results, they also remain valid for any regularization parameters.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 28 of its 42 references that are in the map. First posted on arXiv 2 Oct 2023.

Builds on

28 of 42 references.

References · 28

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  2. 2016Matching Networks for One Shot Learningcited by 2,246
  3. 2014How transferable are features in deep neural networks?cited by 1,216
  4. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 919
  5. 2019Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networkscited by 562
  6. 2013DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognitioncited by 422
  7. 2018Explicit Inductive Bias for Transfer Learning with Convolutional Networkscited by 226
  8. 2020Spectrum Dependent Learning Curves in Kernel Regression and Wide Neural Networkscited by 200
  9. 2018Just Interpolate: Kernel "Ridgeless" Regression Can Generalizecited by 190
  10. 2016Generalization Properties of Learning with Random Featurescited by 163
  11. 2020Benign overfitting in ridge regressioncited by 152
  12. 2019The Convergence Rate of Neural Networks for Learned Functions of Different Frequenciescited by 134
16 more
  1. 2015On the properties of variational approximations of Gibbs posteriorscited by 121
  2. 2021Generalization error of random features and kernel methods: hypercontractivity and kernel matrix concentrationcited by 110
  3. 2020Random Features for Kernel Approximation: A Survey on Algorithms, Theory, and Beyondcited by 107
  4. 2015Less is More: Nystr"om Computational Regularizationcited by 89
  5. 2020Implicit Regularization of Random Feature Modelscited by 54
  6. 2020Approximation Bounds for Random Neural Networks and Reservoir Systemscited by 52
  7. 2023High-dimensional analysis of double descent for linear regression with random projectionscited by 47
  8. 2022Spectrum of inner-product kernel matrices in the polynomial regime and multiple descent phenomenon in kernel ridge regressioncited by 42
  9. 2020Kernel regression in high dimensions: Refined analysis beyond double descentcited by 39
  10. 2019PAC-Bayes Un-Expected Bernstein Inequalitycited by 36
  11. 2022Robust Fine-Tuning of Deep Neural Networks with Hessian-based Generalization Guaranteescited by 34
  12. 2021Optimal Stopping via Randomized Neural Networkscited by 26
  13. 2019Risk bounds for reservoir computingcited by 22
  14. 2022Instance-Dependent Generalization Bounds via Optimal Transportcited by 13
  15. 2023Infinite-dimensional reservoir computingcited by 7
  16. 2022Target alignment in truncated kernel ridge regressioncited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 3; 2025: 3; 2026: 1.

8 papers.

  1. 2023Generalization in Kernel Regression Under Realistic Assumptionscited by 26
  2. 2024Characterizing Overfitting in Kernel Ridgeless Regression Through the Eigenspectrumcited by 13
  3. 2024A Comprehensive Analysis on the Learning Curve in Kernel Ridge Regressioncited by 6
  4. 2025Random Feature Representation Boostingcited by 4
  5. 2025Generalisation and benign over-fitting for linear regression onto random functional covariatescited by 1
  6. 2025Beyond Universal Approximation Theorems: Algorithmic Uniform Approximation by Neural Networks Trained with Noisy Datacited by 1
  7. 2024Tighter Learning Guarantees on Digital Computers via Concentration of Measure on Finite Spacescited by 0
  8. 2026Random Neural Network Expressivity for Non-Linear Partial Differential Equationscited by 0