The Optimality of Kernel Classifiers in Sobolev Space
Not on the map. Why?
Abstract (arXiv)
Kernel methods are widely used in machine learning, especially for classification problems. However, the theoretical analysis of kernel classification is still limited. This paper investigates the statistical performances of kernel classifiers. With some mild assumptions on the conditional probability η(x)=mathbbP(Y=1mid X=x), we derive an upper bound on the classification excess risk of a kernel classifier using recent advances in the theory of kernel regression. We also obtain a minimax lower bound for Sobolev spaces, which shows the optimality of the proposed classifier. Our theoretical results can be extended to the generalization error of overparameterized neural network classifiers. To make our theoretical results more applicable in realistic settings, we also propose a simple method to estimate the interpolation smoothness of 2η(x)-1 and apply the method to real datasets.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 11 of its 44 references that are in the map. First posted on arXiv 2 Feb 2024.
Builds on
11 of 44 references.
References · 11
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
- 2018To understand deep learning we need to understand kernel learningcited by 186
- 2020Evaluation of Neural Architectures Trained with Square Loss vs Cross-Entropy in Classification Taskscited by 137
- 2017Sobolev Norm Learning Rates for Regularized Least-Squares Algorithmcited by 113
- 2018Statistical Optimality of Stochastic Gradient Descent on Hard Learning Problems through Multiple Passescited by 69
- 2023Generalization Ability of Wide Neural Networks on ℝcited by 28
- 2023Mind the spikes: Benign overfitting of kernels and neural networks in fixed dimensioncited by 26
- 2021Understanding Square Loss in Training Overparametrized Neural Network Classifierscited by 14
- 2023Random Smoothing Regularization in Kernel Gradient Descent Learningcited by 6
Built on by
Citing papers per year: 2025: 1.
1 paper.