# The generalization error of max-margin linear classifiers: Benign overfitting and high dimensional asymptotics in the overparametrized regime

arXiv: https://arxiv.org/abs/1911.01544 · HTML: https://mlsanity.com/paper/1911.01544/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Nov 2019.

## Built on by (most cited first, 28 of 28)

- [Hitting the High-Dimensional Notes: An ODE for SGD learning dynamics on GLMs and multi-index models](https://mlsanity.com/paper/2308.08977/) (2023)
- [The Double-Edged Sword of Implicit Bias: Generalization vs. Robustness in ReLU Networks](https://mlsanity.com/paper/2303.01456/) (2023)
- [High-dimensional Analysis of Knowledge Distillation: Weak-to-Strong Generalization and Scaling Laws](https://mlsanity.com/paper/2410.18837/) (2024)
- [Sharp global convergence guarantees for iterative nonconvex optimization: A Gaussian process perspective](https://mlsanity.com/paper/2109.09859/) (2021)
- [Mitigating multiple descents: A model-agnostic framework for risk monotonization](https://mlsanity.com/paper/2205.12937/) (2022)
- [Spurious Correlations in High Dimensional Regression: The Roles of Regularization, Simplicity Bias and Over-Parameterization](https://mlsanity.com/paper/2502.01347/) (2025)
- [Benefits of Early Stopping in Gradient Descent for Overparameterized Logistic Regression](https://mlsanity.com/paper/2502.13283/) (2025)
- [Feature Learning beyond the Lazy-Rich Dichotomy: Insights from Representational Geometry](https://mlsanity.com/paper/2503.18114/) (2025)
- [Preventing Model Collapse Under Overparametrization: Optimal Mixing Ratios for Interpolation Learning and Ridge Regression](https://mlsanity.com/paper/2509.22341/) (2025)
- [Overfitting Behaviour of Gaussian Kernel Ridgeless Regression: Varying Bandwidth or Dimensionality](https://mlsanity.com/paper/2409.03891/) (2024)
- [High-dimensional Analysis of Synthetic Data Selection](https://mlsanity.com/paper/2510.08123/) (2025)
- [Universality of max-margin classifiers](https://mlsanity.com/paper/2310.00176/) (2023)
- [Benign overfitting in leaky ReLU networks with moderate input dimension](https://mlsanity.com/paper/2403.06903/) (2024)
- [A New Central Limit Theorem for the Augmented IPW Estimator: Variance Inflation, Cross-Fit Covariance and Beyond](https://mlsanity.com/paper/2205.10198/) (2022)
- [High-dimensional logistic regression with missing data: Imputation, regularization, and universality](https://mlsanity.com/paper/2410.01093/) (2024)
- [Provable Tempered Overfitting of Minimal Nets and Typical Nets](https://mlsanity.com/paper/2410.19092/) (2024)
- [A Novel Gaussian Min-Max Theorem and its Applications](https://mlsanity.com/paper/2402.07356/) (2024)
- [Quantifying Overfitting along the Regularization Path for Two-Part-Code MDL in Supervised Classification](https://mlsanity.com/paper/2503.02110/) (2025)
- [Eigenvalue distribution of the Neural Tangent Kernel in the quadratic scaling](https://mlsanity.com/paper/2508.20036/) (2025)
- [Optimal Regularization for Performative Learning](https://mlsanity.com/paper/2510.12249/) (2025)
- [RAPTOR: Ridge-Adaptive Logistic Probes](https://mlsanity.com/paper/2602.00158/) (2026)
- [Hard labels sampled from sparse targets mislead rotation invariant algorithms](https://mlsanity.com/paper/2603.20967/) (2026)
- [Benign overfitting beyond prediction: The ordinary least squares interpolator](https://mlsanity.com/paper/2309.15769/) (2023)
- [Benign Overfitting under Learning Rate Conditions for α Sub-exponential Input](https://mlsanity.com/paper/2409.00733/) (2024)
- [Improved generalization bounds for binary linear classification via isoperimetry](https://mlsanity.com/paper/2505.16713/) (2025)
- [One-Bit Quantization for Random Features Models](https://mlsanity.com/paper/2510.16250/) (2025)
- [Double descent for least-squares interpolation on contaminated data: A simulation study](https://mlsanity.com/paper/2605.21494/) (2026)
- [When Stronger Triggers Backfire: A High-Dimensional Theory of Backdoor Attacks](https://mlsanity.com/paper/2605.22481/) (2026)

## Abstract (arXiv)

Modern machine learning classifiers often exhibit vanishing classification error on the training set. They achieve this by learning nonlinear representations of the inputs that maps the data into linearly separable classes. Motivated by these phenomena, we revisit high-dimensional maximum margin classification for linearly separable data. We consider a stylized setting in which data (yi,boldsymbol xi), i≤ n are i.i.d. with boldsymbol xi~mathsfN(boldsymbol 0,boldsymbol Sigma) a p-dimensional Gaussian feature vector, and yi ∈+1,-1 a label whose distribution depends on a linear combination of the covariates langle boldsymbol θ_*,boldsymbol xi rangle. While the Gaussian model might appear extremely simplistic, universality arguments can be used to show that the results derived in this setting also apply to the output of certain nonlinear featurization maps. We consider the proportional asymptotics n,p→∞ with p/n→ psi, and derive exact expressions for the limiting generalization error. We use this theory to derive two results of independent interest: (i) Sufficient conditions on (boldsymbol Sigma,boldsymbol θ_*) for `benign overfitting' that parallel previously derived conditions in the case of linear regression; (ii) An asymptotically exact expression for the generalization error when max-margin classification is used in conjunction with feature vectors produced by random one-layer neural networks.