# Spectral Statistics of the Sample Covariance Matrix for High Dimensional Linear Gaussians

arXiv: https://arxiv.org/abs/2312.05794 · HTML: https://mlsanity.com/paper/2312.05794/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 17 references that are in the map.
- First posted on arXiv 10 Dec 2023.

## Builds on (most cited first, 6 of 6)

- [Learning Without Mixing: Towards A Sharp Analysis of Linear System Identification](https://mlsanity.com/paper/1802.08334/) (2018)
- [Least Squares Regression with Markovian Data: Fundamental Limits and Algorithms](https://mlsanity.com/paper/2006.08916/) (2020)
- [Online Learning of the Kalman Filter with Logarithmic Regret](https://mlsanity.com/paper/2002.05141/) (2020)
- [Linear Systems can be Hard to Learn](https://mlsanity.com/paper/2104.01120/) (2021)
- [High Dimensional Geometry and Limitations in System Identification](https://mlsanity.com/paper/2305.12083/) (2023)
- [From Spectral Theorem to Statistical Independence with Application to System Identification](https://mlsanity.com/paper/2310.10523/) (2023)

## Abstract (arXiv)

Performance of ordinary least squares(OLS) method for the estimation of high dimensional stable state transition matrix A(i.e., spectral radius ρ(A)<1) from a single noisy observed trajectory of the linear time invariant(LTI)footnoteLinear Gaussian (LG) in Markov chain literature system X-:(x0,x1, ldots,xN-1) satisfying beginequation xt+1=Axt+wt, hspace10pt  where  wt thicksim N(0,In), endequation heavily rely on negative moments of the sample covariance matrix: (X-X-*)=sumi=0N-1xixi* and singular values of EX-*, where E is a rectangular Gaussian ensemble E=[w0, ldots, wN-1]. Negative moments requires sharp estimates on all the eigenvalues λ1big(X-X-*big) ≥ ldots ≥ λnbig(X-X-*big) ≥ 0. Leveraging upon recent results on spectral theorem for non-Hermitian operators in citenaeem2023spectral, along with concentration of measure phenomenon and perturbation theory(Gershgorins' and Cauchys' interlacing theorem) we show that only when A=A*, typical order of λjbig(X-X-*big) ∈ big[N-n√N, N+n√Nbig] for all j ∈ [n]. However, in high dimensions when A has only one distinct eigenvalue λ with geometric multiplicity of one, then as soon as eigenvalue leaves complex half unit disc, largest eigenvalue suffers from curse of dimensionality: λ1big(X-X-*big)=Omegabig( lfloorfracNnrfloor e^αλn big), while smallest eigenvalue λnbig(X-X-*big) ∈ (0, N+√N]. Consequently, OLS estimator incurs a phase transition and becomes transient: increasing iteration only worsens estimation error, all of this happening when the dynamics are generated from stable systems.