# High Dimensional Classification via Regularized and Unregularized Empirical Risk Minimization: Precise Error and Optimal Loss

arXiv: https://arxiv.org/abs/1905.13742 · HTML: https://mlsanity.com/paper/1905.13742/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 May 2019.

## Built on by (most cited first, 9 of 9)

- [Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation](https://mlsanity.com/paper/2105.14368/) (2021)
- [Classifying high-dimensional Gaussian mixtures: Where kernel methods fail and neural networks succeed](https://mlsanity.com/paper/2102.11742/) (2021)
- [Universality laws for Gaussian mixtures in generalized linear models](https://mlsanity.com/paper/2302.08933/) (2023)
- [SphereFed: Hyperspherical Federated Learning](https://mlsanity.com/paper/2207.09413/) (2022)
- [Benign Overfitting in Classification: Provably Counter Label Noise with Larger Models](https://mlsanity.com/paper/2206.00501/) (2022)
- [On the Regularization of Autoencoders](https://mlsanity.com/paper/2110.11402/) (2021)
- [Classifying Overlapping Gaussian Mixtures in High Dimensions: From Optimal Classifiers to Neural Nets](https://mlsanity.com/paper/2405.18427/) (2024)
- [Analysis of High-dimensional Gaussian Labeled-unlabeled Mixture Model via Message-passing Algorithm](https://mlsanity.com/paper/2411.19553/) (2024)
- [A Large-Dimensional Analysis of Least-Squares Federated Learning via Generalized Total Variation Minimization](https://mlsanity.com/paper/2610.20075/) (2026)

## Abstract (arXiv)

This article provides, through theoretical analysis, an in-depth understanding of the classification performance of the empirical risk minimization framework, in both ridge-regularized and unregularized cases, when high dimensional data are considered. Focusing on the fundamental problem of separating a two-class Gaussian mixture, the proposed analysis allows for a precise prediction of the classification error for a set of numerous data vectors x ∈ mathbb Rp of sufficiently large dimension p. This precise error depends on the loss function, the number of training samples, and the statistics of the mixture data model. It is shown to hold beyond Gaussian distribution under some additional non-sparsity condition of the data statistics. Building upon this quantitative error analysis, we identify the simple square loss as the optimal choice for high dimensional classification in both ridge-regularized and unregularized cases, regardless of the number of training samples.