# Statistical consistency and asymptotic normality for high-dimensional robust M-estimators

arXiv: https://arxiv.org/abs/1501.00312 · HTML: https://mlsanity.com/paper/1501.00312/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jan 2015.

## Built on by (most cited first, 27 of 27)

- [Differentially private inference via noisy optimization](https://mlsanity.com/paper/2103.11003/) (2021)
- [On Empirical Risk Minimization with Dependent and Heavy-Tailed Data](https://mlsanity.com/paper/2109.02224/) (2021)
- [Asymptotics of Non-Convex Generalized Linear Models in High-Dimensions: A proof of the replica formula](https://mlsanity.com/paper/2502.20003/) (2025)
- [A Kernel-Expanded Stochastic Neural Network](https://mlsanity.com/paper/2201.05319/) (2022)
- [Quantizing Heavy-tailed Data in Statistical Estimation: (Near) Minimax Rates, Covariate Quantization, and Uniform Recovery](https://mlsanity.com/paper/2212.14562/) (2022)
- [Non-Asymptotic Guarantees for Robust Statistical Learning under Infinite Variance Assumption](https://mlsanity.com/paper/2201.03182/) (2022)
- [Gaining Outlier Resistance with Progressive Quantiles: Fast Algorithms and Theoretical Studies](https://mlsanity.com/paper/2112.08471/) (2021)
- [Robust Matrix Completion with Heavy-tailed Noise](https://mlsanity.com/paper/2206.04276/) (2022)
- [Perceptual adjustment queries and an inverted measurement paradigm for low-rank metric learning](https://mlsanity.com/paper/2309.04626/) (2023)
- [A Piecewise Lyapunov Analysis of Sub-quadratic SGD: Applications to Robust and Quantile Regression](https://mlsanity.com/paper/2504.08178/) (2025)
- [Heavy Lasso: sparse penalized regression under heavy-tailed noise via data-augmented soft-thresholding](https://mlsanity.com/paper/2506.07790/) (2025)
- [Robust Variable Selection and Estimation Via Adaptive Elastic Net S-Estimators for Linear Regression](https://mlsanity.com/paper/2107.03325/) (2021)
- [Robust High-Dimensional Regression with Coefficient Thresholding and its Application to Imaging Data Analysis](https://mlsanity.com/paper/2109.14856/) (2021)
- [Metric Entropy-Free Sample Complexity Bounds for Sample Average Approximation in Convex Stochastic Programming](https://mlsanity.com/paper/2401.00664/) (2024)
- [Non-Asymptotic Analysis of Online Local Private Learning with SGD](https://mlsanity.com/paper/2507.07041/) (2025)
- [Statistical Inference on Gradient Flows](https://mlsanity.com/paper/2606.01257/) (2026)
- [An algorithm-based multiple detection influence measure for high dimensional regression using expectile](https://mlsanity.com/paper/2105.12286/) (2021)
- [Markov subsampling based Huber Criterion](https://mlsanity.com/paper/2112.06134/) (2021)
- [An Interpretable and Efficient Infinite-Order Vector Autoregressive Model for High-Dimensional Time Series](https://mlsanity.com/paper/2209.01172/) (2022)
- [Robust High-dimensional Tuning Free Multiple Testing](https://mlsanity.com/paper/2211.11959/) (2022)
- [Retire: Robust Expectile Regression in High Dimensions](https://mlsanity.com/paper/2212.05562/) (2022)
- [The Adaptive τ-Lasso: Robustness and Oracle Properties](https://mlsanity.com/paper/2304.09310/) (2023)
- [Projective Proximal Gradient Descent for A Class of Nonconvex Nonsmooth Optimization Problems: Fast Convergence Without Kurdyka-Lojasiewicz (KL) Property](https://mlsanity.com/paper/2304.10499/) (2023)
- [Robust Analysis for Resilient AI System](https://mlsanity.com/paper/2509.06172/) (2025)
- [Exponential Lasso: robust sparse penalization under heavy-tailed noise and outliers with exponential-type loss](https://mlsanity.com/paper/2511.15332/) (2025)
- [Robust Tensor Regression with Nonconvexity: Algorithmic and Statistical Theory](https://mlsanity.com/paper/2605.07448/) (2026)
- [Demixing Sparse Signals from Nonlinear Observations using Generalized Non-convex Regularization](https://mlsanity.com/paper/2607.10618/) (2026)

## Abstract (arXiv)

We study theoretical properties of regularized robust M-estimators, applicable when data are drawn from a sparse high-dimensional linear model and contaminated by heavy-tailed distributions and/or outliers in the additive errors and covariates. We first establish a form of local statistical consistency for the penalized regression estimators under fairly mild conditions on the error distribution: When the derivative of the loss function is bounded and satisfies a local restricted curvature condition, all stationary points within a constant radius of the true regression vector converge at the minimax rate enjoyed by the Lasso with sub-Gaussian errors. When an appropriate nonconvex regularizer is used in place of an l_1-penalty, we show that such stationary points are in fact unique and equal to the local oracle solution with the correct support---hence, results on asymptotic normality in the low-dimensional case carry over immediately to the high-dimensional setting. This has important implications for the efficiency of regularized nonconvex M-estimators when the errors are heavy-tailed. Our analysis of the local curvature of the loss function also has useful consequences for optimization when the robust regression function and/or regularizer is nonconvex and the objective function possesses stationary points outside the local region. We show that as long as a composite gradient descent algorithm is initialized within a constant radius of the true regression vector, successive iterates will converge at a linear rate to a stationary point within the local region. Furthermore, the global optimum of a convex regularized robust regression function may be used to obtain a suitable initialization. The result is a novel two-step procedure that uses a convex M-estimator to achieve consistency and a nonconvex M-estimator to increase efficiency.