# Heavy-Tailed Principal Component Analysis

arXiv: https://arxiv.org/abs/2603.11308 · HTML: https://mlsanity.com/paper/2603.11308/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 40 references that are in the map.
- First posted on arXiv 11 Mar 2026.

## Builds on (most cited first, 12 of 12)

- [The generalization error of random features regression: Precise asymptotics and double descent curve](https://mlsanity.com/paper/1908.05355/) (2019)
- [Generalisation error in learning with random features and the hidden manifold model](https://mlsanity.com/paper/2002.09339/) (2020)
- [Tensor Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Tensors via Convex Optimization](https://mlsanity.com/paper/1708.04181/) (2017)
- [High-dimensional robust regression under heavy-tailed data: Asymptotics and Universality](https://mlsanity.com/paper/2309.16476/) (2023)
- [Coherence Pursuit: Fast, Simple, and Robust Principal Component Analysis](https://mlsanity.com/paper/1609.04789/) (2016)
- [Classification of Heavy-tailed Features in High Dimensions: a Superstatistical Approach](https://mlsanity.com/paper/2304.02912/) (2023)
- [Spherical Principal Component Analysis](https://mlsanity.com/paper/1903.06877/) (2019)
- [Robust PCA Unrolling Network for Super-resolution Vessel Extraction in X-ray Coronary Angiography](https://mlsanity.com/paper/2204.08466/) (2022)
- [Online Robust Principal Component Analysis with Change Point Detection](https://mlsanity.com/paper/1702.05698/) (2017)
- [Robust PCA for High Dimensional Data based on Characteristic Transformation](https://mlsanity.com/paper/2204.01042/) (2022)
- [Cauchy robust principal component analysis with applications to high-deimensional data sets](https://mlsanity.com/paper/2211.03181/) (2022)
- [Cauchy Principal Component Analysis](https://mlsanity.com/paper/1412.6506/) (2014)

## Abstract (arXiv)

Principal Component Analysis (PCA) is a cornerstone of dimensionality reduction, yet its classical formulation relies critically on second-order moments and is therefore fragile in the presence of heavy-tailed data and impulsive noise. While numerous robust PCA variants have been proposed, most either assume finite variance, rely on sparsity-driven decompositions, or address robustness through surrogate loss functions without a unified treatment of infinite-variance models. In this paper, we study PCA for high-dimensional data generated according to a superstatistical dependent model of the form X = A1/2G, where A is a positive random scalar and G is a Gaussian vector. This framework captures a wide class of heavy-tailed distributions, including multivariate t and sub-Gaussian α-stable laws. We formulate PCA under a logarithmic loss, which remains well defined even when moments do not exist. Our main theoretical result shows that, under this loss, the principal components of the heavy-tailed observations coincide with those obtained by applying standard PCA to the covariance matrix of the underlying Gaussian generator. Building on this insight, we propose robust estimators for this covariance matrix directly from heavy-tailed data and compare them with the empirical covariance and Tyler's scatter estimator. Extensive experiments, including background denoising tasks, demonstrate that the proposed approach reliably recovers principal directions and significantly outperforms classical PCA in the presence of heavy-tailed and impulsive noise, while remaining competitive under Gaussian noise.