mlsanityData to 7 Oct 2026

Random feature approximation for general spectral methods

arXiv 2308.15434 ↗First posted 29 Aug 2023cs.LG stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

Random feature approximation is arguably one of the most popular techniques to speed up kernel methods in large scale algorithms and provides a theoretical approach to the analysis of deep neural networks. We analyze generalization properties for a large class of spectral regularization methods combined with random features, containing kernel methods with implicit regularization such as gradient descent or explicit methods like Tikhonov regularization. For our estimators we obtain optimal learning rates over regularity classes (even for classes that are not included in the reproducing kernel Hilbert space), which are defined through appropriate source conditions. This improves or completes previous results obtained in related settings for specific kernel algorithms.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 21 of its 45 references that are in the map. First posted on arXiv 29 Aug 2023.

Builds on

21 of 45 references.

References · 21

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  2. 2019Towards moderate overparameterization: global convergence guarantees for training shallow neural networkscited by 176
  3. 2016Generalization Properties of Learning with Random Featurescited by 163
  4. 2016Katyusha: The First Direct Acceleration of Stochastic Gradient Methodscited by 134
  5. 2014A Proximal Stochastic Gradient Method with Progressive Variance Reductioncited by 78
  6. 2018Statistical Optimality of Stochastic Gradient Descent on Hard Learning Problems through Multiple Passescited by 69
  7. 2016Optimal Rates For Regularization Of Statistical Inverse Learning Problemscited by 68
  8. 2014Global convergence of the Heavy-ball method for convex optimizationcited by 60
  9. 2018Optimal Rates for Spectral Algorithms with Least-Squares Regression over Hilbert Spacescited by 59
  10. 2020Every Model Learned by Gradient Descent Is Approximately a Kernel Machinecited by 53
  11. 2016Stochastic Heavy Ballcited by 40
  12. 2020Optimal Rates for Averaged Stochastic Gradient Descent under Neural Tangent Kernel Regimecited by 35
9 more
  1. 2018Learning with SGD and Random Featurescited by 33
  2. 2021The Future is Log-Gaussian: ResNets and Their Infinite-Depth-and-Width Limit at Initializationcited by 33
  3. 2023On the Optimality of Misspecified Spectral Algorithmscited by 28
  4. 2023Error Bounds for Learning with Vector-Valued Random Featurescited by 23
  5. 2018Optimal Convergence for Distributed Learning with Stochastic Gradient Methods and Spectral Algorithmscited by 22
  6. 2020Learning to Learn Kernels with Variational Random Featurescited by 18
  7. 2022Bounding the Width of Neural Networks via Coupled Initialization -- A Worst Case Analysiscited by 15
  8. 2019Implicit Regularization of Accelerated Methods in Hilbert Spacescited by 14
  9. 2021Statistical Optimality and Computational Efficiency of Nystr"om Kernel PCAcited by 10

Built on by

Not cited yet.