# Heavy Lasso: sparse penalized regression under heavy-tailed noise via data-augmented soft-thresholding

arXiv: https://arxiv.org/abs/2506.07790 · HTML: https://mlsanity.com/paper/2506.07790/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 41 references that are in the map.
- First posted on arXiv 9 Jun 2025.

## Builds on (most cited first, 10 of 10)

- [Robust Regression via Hard Thresholding](https://mlsanity.com/paper/1506.02428/) (2015)
- [Outlier-robust estimation of a sparse linear model using ℓ1-penalized Huber's M-estimator](https://mlsanity.com/paper/1904.06288/) (2019)
- [Statistical consistency and asymptotic normality for high-dimensional robust M-estimators](https://mlsanity.com/paper/1501.00312/) (2015)
- [Semismooth Newton Coordinate Descent Algorithm for Elastic-Net Penalized Huber Loss Regression and Quantile Regression](https://mlsanity.com/paper/1509.02957/) (2015)
- [Concentration of a sparse Bayesian model with Horseshoe prior in estimating high-dimensional precision matrix](https://mlsanity.com/paper/2406.14269/) (2024)
- [Concentration properties of fractional posterior in 1-bit matrix completion](https://mlsanity.com/paper/2404.08969/) (2024)
- [Scale calibration for high-dimensional robust regression](https://mlsanity.com/paper/1811.02096/) (2018)
- [Rank-based Lasso -- efficient methods for high-dimensional robust model selection](https://mlsanity.com/paper/1905.05876/) (2019)
- [High-dimensional Bayesian Tobit regression for censored response with Horseshoe prior](https://mlsanity.com/paper/2505.08288/) (2025)
- [Ultrahigh-dimensional Robust and Efficient Sparse Regression using Non-Concave Penalized Density Power Divergence](https://mlsanity.com/paper/1802.04906/) (2018)

## Built on by (most cited first, 2 of 2)

- [Exponential Lasso: robust sparse penalization under heavy-tailed noise and outliers with exponential-type loss](https://mlsanity.com/paper/2511.15332/) (2025)
- [Sparse classification with positive-confidence data in high dimensions](https://mlsanity.com/paper/2512.24443/) (2025)

## Abstract (arXiv)

High-dimensional linear regression is a fundamental tool in modern statistics, particularly when the number of predictors exceeds the sample size. The classical Lasso, which relies on the squared loss, performs well under Gaussian noise assumptions but often deteriorates in the presence of heavy-tailed errors or outliers commonly encountered in real data applications such as genomics, finance, and signal processing. To address these challenges, we propose a novel robust regression method, termed Heavy Lasso, which incorporates a loss function inspired by the Student's t-distribution within a Lasso penalization framework. This loss retains the desirable quadratic behavior for small residuals while adaptively downweighting large deviations, thus enhancing robustness to heavy-tailed noise and outliers. Heavy Lasso enjoys computationally efficient by leveraging a data augmentation scheme and a soft-thresholding algorithm, which integrate seamlessly with classical Lasso solvers. Theoretically, we establish non-asymptotic bounds under both ℓ1 and ℓ2  norms, by employing the framework of localized convexity, showing that the Heavy Lasso estimator achieves rates comparable to those of the Huber loss. Extensive numerical studies demonstrate Heavy Lasso's superior performance over classical Lasso and other robust variants, highlighting its effectiveness in challenging noisy settings. Our method is implemented in the R package heavylasso available on Github.