mlsanityData to 7 Oct 2026

Ridgeless Regression with Random Features

arXiv 2205.00477 ↗First posted 1 May 2022cs.LG cs.IT math.ITcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent theoretical studies illustrated that kernel ridgeless regression can guarantee good generalization ability without an explicit regularization. In this paper, we investigate the statistical properties of ridgeless regression with random features and stochastic gradient descent. We explore the effect of factors in the stochastic gradient and random features, respectively. Specifically, random features error exhibits the double-descent curve. Motivated by the theoretical findings, we propose a tunable kernel algorithm that optimizes the spectral density of kernel during training. Our work bridges the interpolation theory and practical algorithm.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 14 of its 32 references that are in the map. First posted on arXiv 1 May 2022.

Builds on

14 of 32 references.

References · 14

  1. 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 667
  2. 2019Benign Overfitting in Linear Regressioncited by 544
  3. 2019Surprises in High-Dimensional Ridgeless Least Squares Interpolationcited by 508
  4. 2019The generalization error of random features regression: Precise asymptotics and double descent curvecited by 410
  5. 2017High-dimensional dynamics of generalization error in neural networkscited by 238
  6. 2018Just Interpolate: Kernel "Ridgeless" Regression Can Generalizecited by 190
  7. 2018To understand deep learning we need to understand kernel learningcited by 186
  8. 2016Generalization Properties of Learning with Random Featurescited by 163
  9. 2019Linearized two-layers neural networks in high dimensioncited by 155
  10. 2019Scaling description of generalization with number of parameters in deep learningcited by 83
  11. 2020Implicit Regularization of Random Feature Modelscited by 54
  12. 2018On Fast Leverage Score Sampling and Optimal Learningcited by 40
2 more
  1. 2018Learning with SGD and Random Featurescited by 33
  2. 2019Automated Spectral Kernel Learningcited by 3

Built on by

Not cited yet.