mlsanityData to 14 Oct 2026

Regularized least squares learning with heavy-tailed noise is minimax optimal

arXiv 2505.14214 ↗First posted 20 May 2025cs.LG math.ST stat.MLcited by 3

Not on the map. Why?

Abstract (arXiv)

This paper examines the performance of ridge regression in reproducing kernel Hilbert spaces in the presence of noise that exhibits a finite number of higher moments. We establish excess risk bounds consisting of subgaussian and polynomial terms based on the well known integral operator framework. The dominant subgaussian component allows to achieve convergence rates that have previously only been derived under subexponential noise - a prevalent assumption in related work from the last two decades. These rates are optimal under standard eigenvalue decay conditions, demonstrating the asymptotic robustness of regularized least squares against heavy-tailed noise. Our derivations are based on a Fuk-Nagaev inequality for Hilbert-space valued random variables.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 23 of its 73 references that are in the map. First posted on arXiv 20 May 2025.

Builds on

23 of 73 references.

References · 23

  1. 2016Generalization Properties of Learning with Random Featurescited by 164
  2. 2019Mean estimation and regression under heavy-tailed distributions--a surveycited by 136
  3. 2019Kernel Instrumental Variable Regressioncited by 118
  4. 2017Sobolev Norm Learning Rates for Regularized Least-Squares Algorithmcited by 114
  5. 2016Optimal Rates For Regularization Of Statistical Inverse Learning Problemscited by 68
  6. 2018Optimal Rates for Spectral Algorithms with Least-Squares Regression over Hilbert Spacescited by 59
  7. 2013Loss minimization and parameter estimation with heavy tailscited by 50
  8. 2022Optimal Rates for Regularized Conditional Mean Embedding Learningcited by 50
  9. 2014Learning Theory for Distribution Regressioncited by 48
  10. 2017Sub-Gaussian estimators of the mean of a random vectorcited by 47
  11. 2014Empirical risk minimization for heavy-tailed lossescited by 37
  12. 2022Learning linear operators: Infinite-dimensional regression as a well-behaved non-compact inverse problemcited by 28
11 more
  1. 2023On the Optimality of Misspecified Spectral Algorithmscited by 28
  2. 2023Towards Optimal Sobolev Norm Rates for the Vector-Valued Regularized Least-Squares Algorithmcited by 22
  3. 2019Beating SGD Saturation with Tail-Averaging and Minibatchingcited by 21
  4. 2023On the Optimality of Misspecified Kernel Ridge Regressioncited by 20
  5. 2021Distribution-Free Robust Linear Regressioncited by 19
  6. 2024On the Saturation Effect of Kernel Ridge Regressioncited by 13
  7. 2024Optimal Rates for Vector-Valued Spectral Regularization Learning Algorithmscited by 12
  8. 2016Optimal rates for the regularized learning algorithms under general source conditioncited by 11
  9. 2024Nonparametric Instrumental Regression via Kernel Methods is Minimax Optimalcited by 8
  10. 2019On Least Squares Estimation under Heteroscedastic and Heavy-Tailed Errorscited by 4
  11. 2025On the Robustness of Kernel Ridge Regression Using the Cauchy Loss Functioncited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Cubit: Token Mixer with Kernel Ridge Regressioncited by 0
  2. 2026On Uniform Error Bounds for Kernel Regression under Non-Gaussian Noisecited by 0
  3. 2026A Smoothed Discrepancy Principle for Random Feature Methods and Neural Networkscited by 0