# Asymptotic Analysis of Empirical Risk Minimization on Entry-wise i.i.d. Heavy-Tailed Data

arXiv: https://arxiv.org/abs/2610.07637 · HTML: https://mlsanity.com/paper/2610.07637/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 1 of its 37 references that are in the map.
- First posted on arXiv 6 Oct 2026.

## Builds on (most cited first, 1 of 1)

- [Universality of empirical risk minimization](https://mlsanity.com/paper/2202.08832/) (2022)

## Abstract (arXiv)

Many real-world datasets exhibit unusually large values far more frequently than predicted by Gaussian models. Heavy-tailed distributions capture this behavior, yet evaluating learning performance under them remains challenging because rare, large feature entries retain non-vanishing effects even in high dimensions. Even in the canonical setting of empirical risk minimization for linear regression with entry-wise i.i.d. symmetric α-stable data, a precise asymptotic characterization of prediction has been lacking. In this work, we introduce a functional order parameter that describes the random effective problem associated with each coefficient. Using the replica method, we fully characterize the generalization error in the proportional high-dimensional limit where the sample size and feature dimension diverge at a fixed ratio. Additionally, this analysis establishes a heavy-tail universality law, scaling laws relating typical errors to prediction reliability, and the Bayes-optimal prediction error. In addition to characterizing the effects of extreme entries on the learning process, our method applies broadly to other systems with persistent local heterogeneity.