mlsanityData to 14 Oct 2026

A Simple Algorithm For Scaling Up Kernel Methods

arXiv 2301.11414 ↗First posted 26 Jan 2023cs.LG math.OCcited by 0

Not on the map. Why?

Abstract (arXiv)

The recent discovery of the equivalence between infinitely wide neural networks (NNs) in the lazy training regime and Neural Tangent Kernels (NTKs) (Jacot et al., 2018) has revived interest in kernel methods. However, conventional wisdom suggests kernel methods are unsuitable for large samples due to their computational complexity and memory requirements. We introduce a novel random feature regression algorithm that allows us (when necessary) to scale to virtually infinite numbers of random features. We illustrate the performance of our method on the CIFAR-10 dataset.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 40 of its 49 references that are in the map. First posted on arXiv 26 Jan 2023.

Builds on

40 of 49 references.

References · 40

  1. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  2. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  3. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  4. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  5. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 927
  6. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 787
  7. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 738
  8. 2018On Lazy Training in Differentiable Programmingcited by 675
  9. 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 672
  10. 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 659
  11. 2017Deep Neural Networks as Gaussian Processescited by 622
  12. 2013Network In Networkcited by 605
28 more
  1. 2018Gradient Descent Finds Global Minima of Deep Neural Networkscited by 574
  2. 2019On Exact Computation with an Infinitely Wide Neural Netcited by 573
  3. 2019Benign Overfitting in Linear Regressioncited by 549
  4. 2019Surprises in High-Dimensional Ridgeless Least Squares Interpolationcited by 513
  5. 2018Gaussian Process Behaviour in Wide Deep Neural Networkscited by 317
  6. 2019Two models of double descent for weak featurescited by 195
  7. 2018To understand deep learning we need to understand kernel learningcited by 187
  8. 2018Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processescited by 167
  9. 2019Graph Neural Tangent Kernel: Fusing Graph Neural Networks with Graph Kernelscited by 161
  10. 2020Finite Versus Infinite Neural Networks: an Empirical Studycited by 157
  11. 2020Benign overfitting in ridge regressioncited by 153
  12. 2019Neural Tangents: Fast and Easy Infinite Neural Networks in Pythoncited by 149
  13. 2016Toward Deeper Understanding of Neural Networks: The Power of Initialization and a Dual View on Expressivitycited by 138
  14. 2018Deep Convolutional Networks as shallow Gaussian Processescited by 138
  15. 2021Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolationcited by 132
  16. 2018Does data interpolation contradict statistical optimality?cited by 95
  17. 2019Harnessing the Power of Infinitely Wide Deep Nets on Small-data Taskscited by 87
  18. 2019Enhanced Convolutional Neural Tangent Kernelscited by 76
  19. 2018A Continuous-Time View of Early Stopping for Least Squarescited by 69
  20. 2018A jamming transition from under- to over-parametrization affects loss landscape and generalizationcited by 67
  21. 2020Neural Kernels Without Tangentscited by 56
  22. 2017SGD Learns the Conjugate Kernel Class of the Networkcited by 54
  23. 2016Faster Kernel Ridge Regression Using Sketching and Preconditioningcited by 44
  24. 2014Scalable Kernel Methods via Doubly Stochastic Gradientscited by 36
  25. 2021Scaling Neural Tangent Kernels via Sketching and Random Featurescited by 32
  26. 2017Diving into the shallows: a computational perspective on large-scale shallow learningcited by 31
  27. 2015Steps Toward Deep Kernel Methods from Infinite Neural Networkscited by 28
  28. 2019The Implicit Regularization of Ordinary Least Squares Ensemblescited by 25

Built on by

Not cited yet.