mlsanityData to 14 Oct 2026

Random feature approximation for general spectral methods

arXiv 2506.16283 ↗First posted 19 Jun 2025stat.ML cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Random feature approximation is arguably one of the most widely used techniques for kernel methods in large-scale learning algorithms. In this work, we analyze the generalization properties of random feature methods, extending previous results for Tikhonov regularization to a broad class of spectral regularization techniques. This includes not only explicit methods but also implicit schemes such as gradient descent and accelerated algorithms like the Heavy-Ball and Nesterov method. Through this framework, we enable a theoretical analysis of neural networks and neural operators through the lens of the Neural Tangent Kernel (NTK) approach trained via gradient descent. For our estimators we obtain optimal learning rates over regularity classes (even for classes that are not included in the reproducing kernel Hilbert space), which are defined through appropriate source conditions. This improves or completes previous results obtained in related settings for specific kernel algorithms.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 24 of its 45 references that are in the map. First posted on arXiv 19 Jun 2025.

Builds on

24 of 45 references.

References · 24

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  2. 2019Towards moderate overparameterization: global convergence guarantees for training shallow neural networkscited by 176
  3. 2016Generalization Properties of Learning with Random Featurescited by 164
  4. 2016Katyusha: The First Direct Acceleration of Stochastic Gradient Methodscited by 135
  5. 2014A Proximal Stochastic Gradient Method with Progressive Variance Reductioncited by 78
  6. 2018Statistical Optimality of Stochastic Gradient Descent on Hard Learning Problems through Multiple Passescited by 69
  7. 2016Optimal Rates For Regularization Of Statistical Inverse Learning Problemscited by 68
  8. 2014Global convergence of the Heavy-ball method for convex optimizationcited by 61
  9. 2018Optimal Rates for Spectral Algorithms with Least-Squares Regression over Hilbert Spacescited by 59
  10. 2020Every Model Learned by Gradient Descent Is Approximately a Kernel Machinecited by 55
  11. 2016Stochastic Heavy Ballcited by 42
  12. 2020Optimal Rates for Averaged Stochastic Gradient Descent under Neural Tangent Kernel Regimecited by 35
12 more
  1. 2018Learning with SGD and Random Featurescited by 33
  2. 2021The Future is Log-Gaussian: ResNets and Their Infinite-Depth-and-Width Limit at Initializationcited by 33
  3. 2023On the Optimality of Misspecified Spectral Algorithmscited by 28
  4. 2023Error Bounds for Learning with Vector-Valued Random Featurescited by 23
  5. 2018Optimal Convergence for Distributed Learning with Stochastic Gradient Methods and Spectral Algorithmscited by 22
  6. 2024An operator learning perspective on parameter-to-observable mapscited by 22
  7. 2020Learning to Learn Kernels with Variational Random Featurescited by 18
  8. 2022Bounding the Width of Neural Networks via Coupled Initialization -- A Worst Case Analysiscited by 15
  9. 2019Implicit Regularization of Accelerated Methods in Hilbert Spacescited by 14
  10. 2021Statistical Optimality and Computational Efficiency of Nystr"om Kernel PCAcited by 10
  11. 2023How many Neurons do we need? A refined Analysis for Shallow Networks trained with Gradient Descentcited by 10
  12. 2024Optimal Convergence Rates for Neural Operatorscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Optimal Rates for Generalization of Gradient Descent Methods with Deep Neural Networkscited by 0
  2. 2026A Smoothed Discrepancy Principle for Random Feature Methods and Neural Networkscited by 0