# Exponential Lasso: robust sparse penalization under heavy-tailed noise and outliers with exponential-type loss

arXiv: https://arxiv.org/abs/2511.15332 · HTML: https://mlsanity.com/paper/2511.15332/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 38 references that are in the map.
- First posted on arXiv 19 Nov 2025.

## Builds on (most cited first, 7 of 7)

- [Statistical consistency and asymptotic normality for high-dimensional robust M-estimators](https://mlsanity.com/paper/1501.00312/) (2015)
- [abess: A Fast Best Subset Selection Library in Python and R](https://mlsanity.com/paper/2110.09697/) (2021)
- [Semismooth Newton Coordinate Descent Algorithm for Elastic-Net Penalized Huber Loss Regression and Quantile Regression](https://mlsanity.com/paper/1509.02957/) (2015)
- [Scale calibration for high-dimensional robust regression](https://mlsanity.com/paper/1811.02096/) (2018)
- [A sparse PAC-Bayesian approach for high-dimensional quantile prediction](https://mlsanity.com/paper/2409.01687/) (2024)
- [Rank-based Lasso -- efficient methods for high-dimensional robust model selection](https://mlsanity.com/paper/1905.05876/) (2019)
- [Heavy Lasso: sparse penalized regression under heavy-tailed noise via data-augmented soft-thresholding](https://mlsanity.com/paper/2506.07790/) (2025)

## Abstract (arXiv)

In high-dimensional statistics, the Lasso is a cornerstone method for simultaneous variable selection and parameter estimation. However, its reliance on the squared loss function renders it highly sensitive to outliers and heavy-tailed noise, potentially leading to unreliable model selection and biased estimates. To address this limitation, we introduce the Exponential Lasso, a novel robust method that integrates an exponential-type loss function within the Lasso framework. This loss function is designed to achieve a smooth trade-off between statistical efficiency under Gaussian noise and robustness against data contamination. Unlike other methods that cap the influence of large residuals, the exponential loss smoothly redescends, effectively downweighting the impact of extreme outliers while preserving near-quadratic behavior for small errors. We establish theoretical guarantees showing that the Exponential Lasso achieves strong statistical convergence rates, matching the classical Lasso under ideal conditions while maintaining its robustness in the presence of heavy-tailed contamination. Computationally, the estimator is optimized efficiently via a Majorization-Minimization (MM) algorithm that iteratively solves a series of weighted Lasso subproblems. Numerical experiments demonstrate that the proposed method is highly competitive, outperforming the classical Lasso in contaminated settings and maintaining strong performance even under Gaussian noise. Our method is implemented in the textttR package textttheavylasso available on Github: https://github.com/tienmt/heavylasso