# Affine Invariant Covariance Estimation for Heavy-Tailed Distributions

arXiv: https://arxiv.org/abs/1902.03086 · HTML: https://mlsanity.com/paper/1902.03086/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Feb 2019.

## Built on by (most cited first, 3 of 3)

- [Distribution-Free Robust Linear Regression](https://mlsanity.com/paper/2102.12919/) (2021)
- [Robustifying Markowitz](https://mlsanity.com/paper/2212.13996/) (2022)
- [Entrywise error bounds for low-rank approximations of kernel matrices](https://mlsanity.com/paper/2405.14494/) (2024)

## Abstract (arXiv)

In this work we provide an estimator for the covariance matrix of a heavy-tailed multivariate distributionWe prove that the proposed estimator widehatS admits an textitaffine-invariant bound of the form [(1-varepsilon) S preccurlyeq widehatS preccurlyeq (1+varepsilon) S]in high probability, where S is the unknown covariance matrix, and preccurlyeq is the positive semidefinite order on symmetric matrices. The result only requires the existence of fourth-order moments, and allows for varepsilon = O(√kappa4 dlog(d/δ)/n) where kappa4 is a measure of kurtosis of the distribution, d is the dimensionality of the space, n is the sample size, and 1-δ is the desired confidence level. More generally, we can allow for regularization with level λ, then d gets replaced with the degrees of freedom number. Denoting cond(S) the condition number of S, the computational cost of the novel estimator is O(d2 n + d3log(cond(S))), which is comparable to the cost of the sample covariance estimator in the statistically interesing regime n ≥ d. We consider applications of our estimator to eigenvalue estimation with relative error, and to ridge regression with heavy-tailed random design.