mlsanityData to 14 Oct 2026

The role of regularization in classification of high-dimensional noisy Gaussian mixture

arXiv 2002.11544 ↗First posted 26 Feb 2020stat.ML cond-mat.dis-nn cs.LGcited by 63

Not on the map. Why?

Abstract (arXiv)

We consider a high-dimensional mixture of two Gaussians in the noisy regime where even an oracle knowing the centers of the clusters misclassifies a small but finite fraction of the points. We provide a rigorous analysis of the generalization error of regularized convex classifiers, including ridge, hinge and logistic regression, in the high-dimensional limit where the number n of samples and their dimension d go to infinity while their ratio is fixed to α= n/d. We discuss surprising effects of the regularization that in some cases allows to reach the Bayes-optimal performances. We also illustrate the interpolation peak at low regularization, and analyze the role of the respective sizes of the two clusters.

Key facts. Cited by 63 papers in the map (data through 14 Oct 2026). First posted on arXiv 26 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 11; 2022: 11; 2023: 8; 2024: 17; 2025: 12; 2026: 4.

63 papers.

Sort:

By area:

  1. 2021Label-Imbalanced and Group-Sensitive Classification under Overparameterizationcited by 74
  2. 2021Classifying high-dimensional Gaussian mixtures: Where kernel methods fail and neural networks succeedcited by 69
  3. 2021A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learningcited by 57
  4. 2021Learning Gaussian Mixtures with Generalised Linear Models: Precise Asymptotics in High-dimensionscited by 54
  5. 2022Neural networks trained with SGD learn distributions of increasing complexitycited by 50
  6. 2024Asymptotics of feature learning in two-layer networks after one gradient-stepcited by 38
  7. 2024A phase transition between positional and semantic learning in a solvable model of dot-product attentioncited by 30
  8. 2023Universality laws for Gaussian mixtures in generalized linear modelscited by 28
  9. 2022Gaussian Universality of Perceptrons with Random Labelscited by 26
  10. 2023Are Gaussian data all you need? Extents and limits of universality in high-dimensional generalized linear estimationcited by 26
  11. 2021Phase Transitions in Transfer Learning for High-Dimensional Perceptronscited by 22
  12. 2021On the Inherent Regularization Effects of Noise Injection During Trainingcited by 18
38 more
  1. 2022Theoretical characterization of uncertainty in high-dimensional linear classificationcited by 18
  2. 2022Learning curves for the multi-class teacher-student perceptroncited by 17
  3. 2024Implicit Optimization Bias of Next-Token Prediction in Linear Modelscited by 14
  4. 2021On the interplay between data structure and loss function in classification problemscited by 13
  5. 2022Homophily modulates double descent generalization in graph convolution networkscited by 13
  6. 2022Fundamental limits to learning closed-form mathematical models from datacited by 12
  7. 2024High-dimensional learning of narrow neural networkscited by 12
  8. 2021Information-Theoretic Characterization of the Generalization Error for Iterative Semi-Supervised Learningcited by 11
  9. 2022Monotonicity and Double Descent in Uncertainty Estimation with Gaussian Processescited by 11
  10. 2023Classification of Heavy-tailed Features in High Dimensions: a Superstatistical Approachcited by 11
  11. 2022Bias-inducing geometries: an exactly solvable data model with fairness implicationscited by 9
  12. 2023Regularized Linear Regression for Binary Classificationcited by 7
  13. 2024A High Dimensional Statistical Model for Adversarial Training: Geometry and Trade-Offscited by 7
  14. 2021The Common Intuition to Transfer Learning Can Win or Lose: Case Studies for Linear Regressioncited by 6
  15. 2024Restoring balance: principled under/oversampling of data for optimal classificationcited by 6
  16. 2021Minimum complexity interpolation in random features modelscited by 5
  17. 2022Benign Overfitting in Classification: Provably Counter Label Noise with Larger Modelscited by 5
  18. 2023High-dimensional Asymptotics of VAEs: Threshold of Posterior Collapse and Dataset-Size Dependence of Rate-Distortion Curvecited by 5
  19. 2023Asymptotic Bayes risk of semi-supervised multitask learning on Gaussian mixturecited by 4
  20. 2023Least Squares Regression Can Exhibit Under-Parameterized Double Descentcited by 4
  21. 2024A Novel Gaussian Min-Max Theorem and its Applicationscited by 4
  22. 2024A replica analysis of under-baggingcited by 4
  23. 2024The Breakdown of Gaussian Universality in Classification of High-dimensional Linear Factor Mixturescited by 4
  24. 2024On the Geometry of Regularization in Adversarial Training: High-Dimensional Asymptotics and Generalization Boundscited by 4
  25. 2025Asymptotic Analysis of Two-Layer Neural Networks after One Gradient Step under Gaussian Mixtures Data with Structurecited by 4
  26. 2022Random Features Model with General Convex Regularization: A Fine Grained Analysis with Precise Asymptotic Learning Curvescited by 3
  27. 2024Asymptotic generalization error of a single-layer graph convolutional networkcited by 3
  28. 2025A statistical physics framework for optimal learningcited by 3
  29. 2023Efficient Learning of Minimax Risk Classifiers in High Dimensionscited by 2
  30. 2025The Effect of Optimal Self-Distillation in Noisy Gaussian Mixture Modelcited by 2
  31. 2025Statistical physics analysis of graph neural networks: Approaching optimality in the contextual stochastic block modelcited by 2
  32. 2025On the Existence of Consistent Adversarial Attacks in High-Dimensional Linear Classificationcited by 2
  33. 2025Generalization performance of narrow one-hidden layer networks in the teacher-student settingcited by 2
  34. 2021Optimal regularizations for data generation with probabilistic graphical modelscited by 1
  35. 2024One-Bit Quantization and Sparsification for Multiclass Linear Classification with Strong Regularizationcited by 1
  36. 2024When resampling/reweighting improves feature learning in imbalanced classification?: A toy-model studycited by 1
  37. 2025Class Imbalance in Anomaly Detection: Learning from an Exactly Solvable Modelcited by 1
  38. 2025Thumb on the Scale: Optimal Loss Weighting in Last Layer Retrainingcited by 1

50 most cited of 63 All 63 (JSON)