mlsanityData to 14 Oct 2026

Classification of Heavy-tailed Features in High Dimensions: a Superstatistical Approach

arXiv 2304.02912 ↗First posted 6 Apr 2023stat.ML cond-mat.dis-nn cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

We characterise the learning of a mixture of two clouds of data points with generic centroids via empirical risk minimisation in the high dimensional regime, under the assumptions of generic convex loss and convex regularisation. Each cloud of data points is obtained via a double-stochastic process, where the sample is obtained from a Gaussian distribution whose variance is itself a random parameter sampled from a scalar distribution varrho. As a result, our analysis covers a large family of data distributions, including the case of power-law-tailed distributions with no covariance, and allows us to test recent "Gaussian universality" claims. We study the generalisation performance of the obtained estimator, we analyse the role of regularisation, and we analytically characterise the separability transition.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 25 of its 72 references that are in the map. First posted on arXiv 6 Apr 2023.

Builds on

25 of 72 references.

References · 25

  1. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 927
  2. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 719
  3. 2019Surprises in High-Dimensional Ridgeless Least Squares Interpolationcited by 513
  4. 2019The generalization error of random features regression: Precise asymptotics and double descent curvecited by 413
  5. 2020Generalisation error in learning with random features and the hidden manifold modelcited by 127
  6. 2021Learning curves of generic features maps for realistic datasets with a teacher-student modelcited by 126
  7. 2018Does data interpolation contradict statistical optimality?cited by 95
  8. 2022Neural Collapse: A Review on Modelling Principles and Generalizationcited by 95
  9. 2019Limitations of Lazy Training of Two-layers Neural Networkscited by 87
  10. 2018A modern maximum-likelihood theory for high-dimensional logistic regressioncited by 81
  11. 2019A Model of Double Descent for High-dimensional Binary Linear Classificationcited by 81
  12. 2022Universality of empirical risk minimizationcited by 75
13 more
  1. 2020The role of regularization in classification of high-dimensional noisy Gaussian mixturecited by 63
  2. 2020What Do Neural Networks Learn When Trained With Random Labels?cited by 59
  3. 2021Learning Gaussian Mixtures with Generalised Linear Models: Precise Asymptotics in High-dimensionscited by 54
  4. 2020Random Matrix Theory Proves that Deep Learning Representations of GAN-data Behave as Gaussian Mixturescited by 47
  5. 2021Graph-based Approximate Message Passing Iterationscited by 35
  6. 2023Universality laws for Gaussian mixtures in generalized linear modelscited by 28
  7. 2022Gaussian Universality of Perceptrons with Random Labelscited by 26
  8. 2023Are Gaussian data all you need? Extents and limits of universality in high-dimensional generalized linear estimationcited by 26
  9. 2020Asymptotic Errors for Teacher-Student Convex Generalized Linear Models (or : How to Prove Kabashima's Replica Formula)cited by 23
  10. 2019Asymptotic Bayes risk for Gaussian mixture in a semi-supervised settingcited by 15
  11. 2020The Asymptotic Distribution of the MLE in High-dimensional Logistic Models: Arbitrary Covariancecited by 15
  12. 2020Wide flat minima and optimal generalization in classifying high-dimensional Gaussian mixturescited by 9
  13. 2016Cox process representation and inference for stochastic reaction-diffusion processescited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 4; 2025: 4; 2026: 2.

11 papers.

  1. 2024Asymptotics of feature learning in two-layer networks after one gradient-stepcited by 38
  2. 2023High-dimensional robust regression under heavy-tailed data: Asymptotics and Universalitycited by 15
  3. 2025A solvable model of learning generative diffusion: theory and insightscited by 14
  4. 2024High-dimensional learning of narrow neural networkscited by 12
  5. 2025Double descent in quantum kernel methodscited by 9
  6. 2024A High Dimensional Statistical Model for Adversarial Training: Geometry and Trade-Offscited by 7
  7. 2024On the Geometry of Regularization in Adversarial Training: High-Dimensional Asymptotics and Generalization Boundscited by 4
  8. 2025A statistical physics framework for optimal learningcited by 3
  9. 2025On the Existence of Consistent Adversarial Attacks in High-Dimensional Linear Classificationcited by 2
  10. 2026Spectral phase transitions and trainability in neural network learning dynamicscited by 2
  11. 2026Heavy-Tailed Principal Component Analysiscited by 0