# Moving Beyond Sub-Gaussianity in High-Dimensional Statistics: Applications in Covariance Estimation and Linear Regression

arXiv: https://arxiv.org/abs/1804.02605 · HTML: https://mlsanity.com/paper/1804.02605/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Apr 2018.

## Built on by (most cited first, 30 of 42)

- [A unifying tutorial on Approximate Message Passing](https://mlsanity.com/paper/2105.02180/) (2021)
- [A Theory of Non-Linear Feature Learning with One Gradient Step in Two-Layer Neural Networks](https://mlsanity.com/paper/2310.07891/) (2023)
- [Reinforcement learning for linear-convex models with jumps via stability analysis of feedback controls](https://mlsanity.com/paper/2104.09311/) (2021)
- [Outlier-Robust Learning of Ising Models Under Dobrushin's Condition](https://mlsanity.com/paper/2102.02171/) (2021)
- [Asymmetric Heavy Tails and Implicit Bias in Gaussian Noise Injections](https://mlsanity.com/paper/2102.07006/) (2021)
- [Automatic Change-Point Detection in Time Series via Deep Learning](https://mlsanity.com/paper/2211.03860/) (2022)
- [Variational Inference in high-dimensional linear regression](https://mlsanity.com/paper/2104.12232/) (2021)
- [Quantizing Heavy-tailed Data in Statistical Estimation: (Near) Minimax Rates, Covariate Quantization, and Uniform Recovery](https://mlsanity.com/paper/2212.14562/) (2022)
- [Near Optimal Private and Robust Linear Regression](https://mlsanity.com/paper/2301.13273/) (2023)
- [Sharper Sub-Weibull Concentrations](https://mlsanity.com/paper/2102.02450/) (2021)
- [On the Mechanisms of Weak-to-Strong Generalization: A Theoretical Perspective](https://mlsanity.com/paper/2505.18346/) (2025)
- [High-dimensional Functional Graphical Model Structure Learning via Neighborhood Selection Approach](https://mlsanity.com/paper/2105.02487/) (2021)
- [General Tail Bounds for Non-Smooth Stochastic Mirror Descent](https://mlsanity.com/paper/2312.07142/) (2023)
- [Stochastic Saddle Point Problems with Decision-Dependent Distributions](https://mlsanity.com/paper/2201.02313/) (2022)
- [Time-Uniform Confidence Spheres for Means of Random Vectors](https://mlsanity.com/paper/2311.08168/) (2023)
- [Bayesian neural network unit priors and generalized Weibull-tail property](https://mlsanity.com/paper/2110.02885/) (2021)
- [Double Robust Semi-Supervised Inference for the Mean: Selection Bias under MAR Labeling with Decaying Overlap](https://mlsanity.com/paper/2104.06667/) (2021)
- [Cost-Driven Representation Learning for Linear Quadratic Gaussian Control: Part I](https://mlsanity.com/paper/2212.14511/) (2022)
- [On the best approximation by finite Gaussian mixtures](https://mlsanity.com/paper/2404.08913/) (2024)
- [Convergence of Implicit Gradient Descent for Training Two-Layer Physics-Informed Neural Networks](https://mlsanity.com/paper/2407.02827/) (2024)
- [Nonconvex Stochastic Scaled-Gradient Descent and Generalized Eigenvector Problems](https://mlsanity.com/paper/2112.14738/) (2021)
- [A Framework for Overparameterized Learning](https://mlsanity.com/paper/2205.13507/) (2022)
- [Convergence of Stochastic Gradient Methods for Wide Two-Layer Physics-Informed Neural Networks for the Poisson Equation](https://mlsanity.com/paper/2508.21571/) (2025)
- [Distributed Semi-Supervised Sparse Statistical Inference](https://mlsanity.com/paper/2306.10395/) (2023)
- [Concentration inequalities for high-dimensional linear processes with dependent innovations](https://mlsanity.com/paper/2307.12395/) (2023)
- [A General Reduction for High-Probability Analysis with General Light-Tailed Distributions](https://mlsanity.com/paper/2403.02873/) (2024)
- [Weighted Random Dot Product Graphs](https://mlsanity.com/paper/2505.03649/) (2025)
- [Robust Inference for High-Dimensional Linear Models via Residual Randomization](https://mlsanity.com/paper/2106.07717/) (2021)
- [Improved Learning Rates for Stochastic Optimization](https://mlsanity.com/paper/2107.08686/) (2021)
- [Inference for Change Points in High Dimensional Mean Shift Models](https://mlsanity.com/paper/2107.09150/) (2021)

## Abstract (arXiv)

Concentration inequalities form an essential toolkit in the study of high dimensional (HD) statistical methods. Most of the relevant statistics literature in this regard is based on sub-Gaussian or sub-exponential tail assumptions. In this paper, we first bring together various probabilistic inequalities for sums of independent random variables under much more general exponential type (namely sub-Weibull) tail assumptions. These results extract a part sub-Gaussian tail behavior in finite samples, matching the asymptotics governed by the central limit theorem, and are compactly represented in terms of a new Orlicz quasi-norm - the Generalized Bernstein-Orlicz norm - that typifies such tail behaviors. We illustrate the usefulness of these inequalities through the analysis of four fundamental problems in HD statistics. In the first two problems, we study the rate of convergence of the sample covariance matrix in terms of the maximum elementwise norm and the maximum k-sub-matrix operator norm which are key quantities of interest in bootstrap, HD covariance matrix estimation and HD inference. The third example concerns the restricted eigenvalue condition, required in HD linear regression, which we verify for all sub-Weibull random vectors through a unified analysis, and also prove a more general result related to restricted strong convexity in the process. In the final example, we consider the Lasso estimator for linear regression and establish its rate of convergence under much weaker than usual tail assumptions (on the errors as well as the covariates), while also allowing for misspecified models and both fixed and random design. To our knowledge, these are the first such results for Lasso obtained in this generality. The common feature in all our results over all the examples is that the convergence rates under most exponential tails match the usual ones under sub-Gaussian assumptions.