Spectral Statistics of the Sample Covariance Matrix for High Dimensional Linear Gaussians

Performance of ordinary least squares(OLS) method for the estimation of high dimensional stable state transition matrix A(i.e., spectral radius ρ(A)<1) from a single noisy observed trajectory of the linear time invariant(LTI)footnoteLinear Gaussian (LG) in Markov chain literature system X-:(x0,x1, ldots,xN-1) satisfying beginequation xt+1=Axt+wt, hspace10pt where wt thicksim N(0,In), endequation heavily rely on negative moments of the sample covariance matrix: (X-X-*)=sumi=0N-1xixi* and singular values of EX-*, where E is a rectangular Gaussian ensemble E=[w0, ldots, wN-1]. Negative moments requires sharp estimates on all the eigenvalues λ1big(X-X-*big) ≥ ldots ≥ λnbig(X-X-*big) ≥ 0. Leveraging upon recent results on spectral theorem for non-Hermitian operators in citenaeem2023spectral, along with concentration of measure phenomenon and perturbation theory(Gershgorins' and Cauchys' interlacing theorem) we show that only when A=A*, typical order of λjbig(X-X-*big) ∈ big[N-n√N, N+n√Nbig] for all j ∈ [n]. However, in high dimensions when A has only one distinct eigenvalue λ with geometric multiplicity of one, then as soon as eigenvalue leaves complex half unit disc, largest eigenvalue suffers from curse of dimensionality: λ1big(X-X-*big)=Omegabig( lfloorfracNnrfloor e^αλn big), while smallest eigenvalue λnbig(X-X-*big) ∈ (0, N+√N]. Consequently, OLS estimator incurs a phase transition and becomes transient: increasing iteration only worsens estimation error, all of this happening when the dynamics are generated from stable systems.