# Characterization of Gaussian Universality Breakdown in High-Dimensional Empirical Risk Minimization

arXiv: https://arxiv.org/abs/2604.03146 · HTML: https://mlsanity.com/paper/2604.03146/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 47 references that are in the map.
- First posted on arXiv 3 Apr 2026.

## Builds on (most cited first, 17 of 17)

- [High-Dimensional Asymptotics of Prediction: Ridge Regression and Classification](https://mlsanity.com/paper/1507.03003/) (2015)
- [Learning curves of generic features maps for realistic datasets with a teacher-student model](https://mlsanity.com/paper/2102.08127/) (2021)
- [The Gaussian equivalence of generative models for learning with shallow neural networks](https://mlsanity.com/paper/2006.14709/) (2020)
- [A Random Matrix Approach to Neural Networks](https://mlsanity.com/paper/1702.05419/) (2017)
- [Universality of empirical risk minimization](https://mlsanity.com/paper/2202.08832/) (2022)
- [Learning Gaussian Mixtures with Generalised Linear Models: Precise Asymptotics in High-dimensions](https://mlsanity.com/paper/2106.03791/) (2021)
- [Random Matrix Theory Proves that Deep Learning Representations of GAN-data Behave as Gaussian Mixtures](https://mlsanity.com/paper/2001.08370/) (2020)
- [A framework to characterize performance of LASSO algorithms](https://mlsanity.com/paper/1303.7291/) (2013)
- [Universality laws for Gaussian mixtures in generalized linear models](https://mlsanity.com/paper/2302.08933/) (2023)
- [Gaussian Universality of Perceptrons with Random Labels](https://mlsanity.com/paper/2205.13303/) (2022)
- [Are Gaussian data all you need? Extents and limits of universality in high-dimensional generalized linear estimation](https://mlsanity.com/paper/2302.08923/) (2023)
- [Universality laws for randomized dimension reduction, with applications](https://mlsanity.com/paper/1511.09433/) (2015)
- [High-dimensional robust regression under heavy-tailed data: Asymptotics and Universality](https://mlsanity.com/paper/2309.16476/) (2023)
- [When does Gaussian equivalence fail and how to fix it: Non-universal behavior of random features with quadratic scaling](https://mlsanity.com/paper/2512.03325/) (2025)
- [A Novel Gaussian Min-Max Theorem and its Applications](https://mlsanity.com/paper/2402.07356/) (2024)
- [The Breakdown of Gaussian Universality in Classification of High-dimensional Linear Factor Mixtures](https://mlsanity.com/paper/2410.05609/) (2024)
- [Universality of High-Dimensional Logistic Regression and a Novel CGMT under Dependence with Applications to Data Augmentation](https://mlsanity.com/paper/2502.15752/) (2025)

## Built on by (most cited first, 1 of 1)

- [The sharp SAT/UNSAT phase transition in random ellipsoid fitting](https://mlsanity.com/paper/2608.10184/) (2026)

## Abstract (arXiv)

We study high-dimensional convex empirical risk minimization (ERM) under general non-Gaussian data designs. By heuristically extending the Convex Gaussian Min-Max Theorem (CGMT) to non-Gaussian settings, we derive an asymptotic min-max characterization of key statistics, enabling approximation of the mean μ_hatθ and covariance C_hatθ of the ERM estimator hatθ. Specifically, under a concentration assumption on the data matrix and standard regularity conditions on the loss and regularizer, we show that for a test covariate x independent of the training data, the projection hatθ^top x approximately follows the convolution of the generally non-Gaussian distribution of μ_hatθ^top x with an independent centered Gaussian variable of variance tr(C_hatθ mathbbE[xx^top]). This result clarifies the scope and limits of Gaussian universality for ERMs. Numerical simulations across diverse losses and models are provided to validate our theoretical predictions and qualitative insights.