On Kernel Regression with Data-Dependent Kernels
Not on the map. Why?
Abstract (arXiv)
The primary hyperparameter in kernel regression (KR) is the choice of kernel. In most theoretical studies of KR, one assumes the kernel is fixed before seeing the training data. Under this assumption, it is known that the optimal kernel is equal to the prior covariance of the target function. In this note, we consider KR in which the kernel may be updated after seeing the training data. We point out that an analogous choice of kernel using the posterior of the target function is optimal in this setting. Connections to the view of deep neural networks as data-dependent kernel learners are discussed.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 10 of its 15 references that are in the map. First posted on arXiv 4 Sep 2022.
Builds on
10 of 15 references.
References · 10
- 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
- 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 732
- 2017Deep Neural Networks as Gaussian Processescited by 617
- 2018Gaussian Process Behaviour in Wide Deep Neural Networkscited by 314
- 2020Spectrum Dependent Learning Curves in Kernel Regression and Wide Neural Networkscited by 200
- 2020Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernelcited by 185
- 2021Neural Networks as Kernel Learners: The Silent Alignment Effectcited by 106
- 2020Kernel Alignment Risk Estimator: Risk Prediction from Training Datacited by 54
- 2022Limitations of the NTK for Understanding Generalization in Deep Learningcited by 34
- 2021Properties of the After Kernelcited by 27
Built on by
Citing papers per year: 2023: 1.
1 paper.