mlsanityData to 14 Oct 2026

Initialization-Dependent Sample Complexity of Linear Predictors and Neural Networks

arXiv 2305.16475 ↗First posted 25 May 2023cs.LG stat.MLcited by 4

Not on the map. Why?

Abstract (arXiv)

We provide several new results on the sample complexity of vector-valued linear predictors (parameterized by a matrix), and more generally neural networks. Focusing on size-independent bounds, where only the Frobenius norm distance of the parameters from some fixed reference matrix W0 is controlled, we show that the sample complexity behavior can be surprisingly different than what we may expect considering the well-studied setting of scalar-valued linear predictors. This also leads to new sample complexity bounds for feed-forward neural networks, tackling some open questions in the literature, and establishing a new convex linear prediction problem that is provably learnable without uniform convergence.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 13 of its 22 references that are in the map. First posted on arXiv 25 May 2023.

Builds on

13 of 22 references.

References · 13

  1. 2017Spectrally-normalized margin bounds for neural networkscited by 686
  2. 2019Gradient Descent Maximizes the Margin of Homogeneous Neural Networkscited by 320
  3. 2017A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networkscited by 314
  4. 2015Norm-Based Capacity Control in Neural Networkscited by 299
  5. 2017Size-Independent Sample Complexity of Neural Networkscited by 290
  6. 2019Uniform convergence may be unable to explain generalization in deep learningcited by 224
  7. 2016A vector-contraction inequality for Rademacher complexitiescited by 142
  8. 2018On the Power of Over-parametrization in Neural Networks with Quadratic Activationcited by 102
  9. 2016Generalization of ERM in Stochastic Convex Optimization: The Dimension Strikes Backcited by 37
  10. 2019Generalization Bounds for Neural Networks via Approximate Description Lengthcited by 15
  11. 2022The Sample Complexity of One-Hidden-Layer Neural Networkscited by 10
  12. 2022Max-Margin Works while Large Margin Fails: Generalization without Uniform Convergencecited by 6
1 more
  1. 2022On the Sample Complexity of Two-Layer Networks: Lipschitz vs. Element-Wise Lipschitz Activationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 1.

4 papers.

  1. 2024The Dimension Strikes Back with Gradients: Generalization of Gradient Methods in Stochastic Convex Optimizationcited by 6
  2. 2025Multiclass Loss Geometry Matters for Generalization of Gradient Descent in Separable Classificationcited by 2
  3. 2024Complexity of Vector-valued Prediction: From Linear Models to Stochastic Convex Optimizationcited by 1
  4. 2026Towards Initialization-dependent and Non-vacuous Generalization Bounds for Overparameterized Shallow Neural Networkscited by 0