mlsanityData to 7 Oct 2026

Moving Beyond Sub-Gaussianity in High-Dimensional Statistics: Applications in Covariance Estimation and Linear Regression

arXiv 1804.02605 ↗First posted 8 Apr 2018math.ST stat.ME stat.MLcited by 42

Not on the map. Why?

Abstract (arXiv)

Concentration inequalities form an essential toolkit in the study of high dimensional (HD) statistical methods. Most of the relevant statistics literature in this regard is based on sub-Gaussian or sub-exponential tail assumptions. In this paper, we first bring together various probabilistic inequalities for sums of independent random variables under much more general exponential type (namely sub-Weibull) tail assumptions. These results extract a part sub-Gaussian tail behavior in finite samples, matching the asymptotics governed by the central limit theorem, and are compactly represented in terms of a new Orlicz quasi-norm - the Generalized Bernstein-Orlicz norm - that typifies such tail behaviors. We illustrate the usefulness of these inequalities through the analysis of four fundamental problems in HD statistics. In the first two problems, we study the rate of convergence of the sample covariance matrix in terms of the maximum elementwise norm and the maximum k-sub-matrix operator norm which are key quantities of interest in bootstrap, HD covariance matrix estimation and HD inference. The third example concerns the restricted eigenvalue condition, required in HD linear regression, which we verify for all sub-Weibull random vectors through a unified analysis, and also prove a more general result related to restricted strong convexity in the process. In the final example, we consider the Lasso estimator for linear regression and establish its rate of convergence under much weaker than usual tail assumptions (on the errors as well as the covariates), while also allowing for misspecified models and both fixed and random design. To our knowledge, these are the first such results for Lasso obtained in this generality. The common feature in all our results over all the examples is that the convergence rates under most exponential tails match the usual ones under sub-Gaussian assumptions.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Apr 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 5; 2023: 7; 2024: 5; 2025: 3; 2026: 9.

42 papers.

Sort:

By area:

  1. 2021A unifying tutorial on Approximate Message Passingcited by 49
  2. 2023A Theory of Non-Linear Feature Learning with One Gradient Step in Two-Layer Neural Networkscited by 46
  3. 2021Reinforcement learning for linear-convex models with jumps via stability analysis of feedback controlscited by 21
  4. 2021Outlier-Robust Learning of Ising Models Under Dobrushin's Conditioncited by 14
  5. 2021Asymmetric Heavy Tails and Implicit Bias in Gaussian Noise Injectionscited by 12
  6. 2022Automatic Change-Point Detection in Time Series via Deep Learningcited by 12
  7. 2021Variational Inference in high-dimensional linear regressioncited by 10
  8. 2023Near Optimal Private and Robust Linear Regressioncited by 9
  9. 2021Sharper Sub-Weibull Concentrationscited by 8
  10. 2021High-dimensional Functional Graphical Model Structure Learning via Neighborhood Selection Approachcited by 7
  11. 2022Quantizing Heavy-tailed Data in Statistical Estimation: (Near) Minimax Rates, Covariate Quantization, and Uniform Recoverycited by 7
  12. 2023General Tail Bounds for Non-Smooth Stochastic Mirror Descentcited by 7
30 more
  1. 2025On the Mechanisms of Weak-to-Strong Generalization: A Theoretical Perspectivecited by 7
  2. 2022Stochastic Saddle Point Problems with Decision-Dependent Distributionscited by 6
  3. 2023Time-Uniform Confidence Spheres for Means of Random Vectorscited by 6
  4. 2021Bayesian neural network unit priors and generalized Weibull-tail propertycited by 5
  5. 2021Double Robust Semi-Supervised Inference for the Mean: Selection Bias under MAR Labeling with Decaying Overlapcited by 4
  6. 2022Cost-Driven Representation Learning for Linear Quadratic Gaussian Control: Part Icited by 3
  7. 2024On the best approximation by finite Gaussian mixturescited by 3
  8. 2024Convergence of Implicit Gradient Descent for Training Two-Layer Physics-Informed Neural Networkscited by 3
  9. 2021Nonconvex Stochastic Scaled-Gradient Descent and Generalized Eigenvector Problemscited by 2
  10. 2022A Framework for Overparameterized Learningcited by 2
  11. 2025Convergence of Stochastic Gradient Methods for Wide Two-Layer Physics-Informed Neural Networks for the Poisson Equationcited by 2
  12. 2023Distributed Semi-Supervised Sparse Statistical Inferencecited by 1
  13. 2023Concentration inequalities for high-dimensional linear processes with dependent innovationscited by 1
  14. 2024A General Reduction for High-Probability Analysis with General Light-Tailed Distributionscited by 1
  15. 2025Weighted Random Dot Product Graphscited by 1
  16. 2021Robust Inference for High-Dimensional Linear Models via Residual Randomizationcited by 0
  17. 2021Improved Learning Rates for Stochastic Optimizationcited by 0
  18. 2021Inference for Change Points in High Dimensional Mean Shift Modelscited by 0
  19. 2023Estimation of sparse linear regression coefficients under L-subexponential covariatescited by 0
  20. 2024Exponential tilting of subweibull distributionscited by 0
  21. 2024Convex Estimation of Gaussian Graphical Regression Models with Covariatescited by 0
  22. 2026Tail-Aware Information-Theoretic Bounds for LLM Alignment under Heavy-Tailed Rewardscited by 0
  23. 2026Feature Learning in Linear-Width Two-Layer Networks: Two vs. One Step of Gradient Descentcited by 0
  24. 2026Clipping Makes Distributed and Federated Asynchronous SGD Robust to Stragglerscited by 0
  25. 2026Empirical Bayes for correlated Gaussian sequence modelcited by 0
  26. 2026High-Dimensional Nonparametric Change-Point Detection via Low-Rank Degree-Three Density Projectioncited by 0
  27. 2026Restricted Eigenvalues Beyond Gaussian Width: Threshold Occupancy under Heavy Tailscited by 0
  28. 2026High-Probability Guarantees for SGD under β-Heavy-Tailed Gradient Noisecited by 0
  29. 2026Benign Overfitting for General Norms and Distributionscited by 0
  30. 2026Search Dimension in Unlabeled Projection Pursuit: A Scaling Law for Subspace Restrictioncited by 0