mlsanityData to 7 Oct 2026

Harmful Overfitting in Sobolev Spaces

arXiv 2602.00825 ↗First posted 31 Jan 2026stat.ML cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Motivated by recent work on benign overfitting in overparameterized machine learning, we study the generalization behavior of functions in Sobolev spaces Wk, p(ℝd) that perfectly fit a noisy training data set. Under assumptions of label noise and sufficient regularity in the data distribution, we show that approximately norm-minimizing interpolators, which are canonical solutions selected by smoothness bias, exhibit harmful overfitting: even as the training sample size n → ∞, the generalization error remains bounded below by a positive constant with high probability. Our results hold for arbitrary values of p ∈ [1, ∞), in contrast to prior results studying the Hilbert space case (p = 2) using kernel methods. Our proof uses a geometric argument which identifies harmful neighborhoods of the training data using Sobolev inequalities.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 25 of its 35 references that are in the map. First posted on arXiv 31 Jan 2026.

Builds on

25 of 35 references.

References · 25

  1. 2019Benign Overfitting in Linear Regressioncited by 544
  2. 2019Surprises in High-Dimensional Ridgeless Least Squares Interpolationcited by 508
  3. 2019Harmless interpolation of noisy data in regressioncited by 121
  4. 2022Benign Overfitting in Two-layer Convolutional Neural Networkscited by 90
  5. 2021Benign Overfitting of Constant-Stepsize SGD for Linear Regressioncited by 71
  6. 2021Uniform Convergence of Interpolators: Gaussian Width, Norm Bounds, and Benign Overfittingcited by 54
  7. 2018Consistency of Interpolation with Laplace Kernels is a High-Dimensional Phenomenoncited by 44
  8. 2022The Implicit Bias of Benign Overfittingcited by 38
  9. 2023Benign Overfitting for Two-layer ReLU Convolutional Neural Networkscited by 34
  10. 2023Benign Overfitting and Grokking in ReLU Networks for XOR Cluster Datacited by 32
  11. 2023Benign Overfitting in Linear Classifiers and Leaky ReLU Networks from KKT Conditions for Margin Maximizationcited by 29
  12. 2020Binary Classification of Gaussian Mixtures: Abundance of Support Vectors, Benign Overfitting and Regularizationcited by 27
13 more
  1. 2023Mind the spikes: Benign overfitting of kernels and neural networks in fixed dimensioncited by 26
  2. 2021Foolish Crowds Support Benign Overfittingcited by 24
  3. 2023Kernel interpolation generalizes poorlycited by 16
  4. 2023From Tempered to Benign Overfitting in ReLU Neural Networkscited by 16
  5. 2022On the Inconsistency of Kernel Ridgeless Regression in Fixed Dimensionscited by 13
  6. 2024Characterizing Overfitting in Kernel Ridgeless Regression Through the Eigenspectrumcited by 13
  7. 2023Noisy Interpolation Learning with Shallow Univariate ReLU Networkscited by 11
  8. 2024Benign Overfitting in Single-Head Attentioncited by 11
  9. 2015An extension of McDiarmid's inequalitycited by 8
  10. 2023Training shallow ReLU networks on noisy data using hinge loss: when do we overfit and is it benign?cited by 7
  11. 2024Benign overfitting in leaky ReLU networks with moderate input dimensioncited by 5
  12. 2024Provable Tempered Overfitting of Minimal Nets and Typical Netscited by 5
  13. 2025Beyond Benign Overfitting in Nadaraya-Watson Interpolatorscited by 2

Built on by

Not cited yet.