# The Trimmed Lasso: Sparsity and Robustness

arXiv: https://arxiv.org/abs/1708.04527 · HTML: https://mlsanity.com/paper/1708.04527/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Aug 2017.

## Built on by (most cited first, 7 of 7)

- [Contextual Feature Selection with Conditional Stochastic Gates](https://mlsanity.com/paper/2312.14254/) (2023)
- [Principal Component Hierarchy for Sparse Quadratic Programs](https://mlsanity.com/paper/2105.12022/) (2021)
- [Sparse Centroid-Encoder: A Nonlinear Model for Feature Selection](https://mlsanity.com/paper/2201.12910/) (2022)
- [Feature Selection using Sparse Adaptive Bottleneck Centroid-Encoder](https://mlsanity.com/paper/2306.04795/) (2023)
- [Sparse Linear Centroid-Encoder: A Convex Method for Feature Selection](https://mlsanity.com/paper/2306.04824/) (2023)
- [Exact identification of nonlinear dynamical systems by Trimmed Lasso](https://mlsanity.com/paper/2308.01891/) (2023)
- [An integrated perspective of robustness in regression through the lens of the bias-variance trade-off](https://mlsanity.com/paper/2407.10418/) (2024)

## Abstract (arXiv)

Nonconvex penalty methods for sparse modeling in linear regression have been a topic of fervent interest in recent years. Herein, we study a family of nonconvex penalty functions that we call the trimmed Lasso and that offers exact control over the desired level of sparsity of estimators. We analyze its structural properties and in doing so show the following: 1) Drawing parallels between robust statistics and robust optimization, we show that the trimmed-Lasso-regularized least squares problem can be viewed as a generalized form of total least squares under a specific model of uncertainty. In contrast, this same model of uncertainty, viewed instead through a robust optimization lens, leads to the convex SLOPE (or OWL) penalty. 2) Further, in relating the trimmed Lasso to commonly used sparsity-inducing penalty functions, we provide a succinct characterization of the connection between trimmed-Lasso- like approaches and penalty functions that are coordinate-wise separable, showing that the trimmed penalties subsume existing coordinate-wise separable penalties, with strict containment in general. 3) Finally, we describe a variety of exact and heuristic algorithms, both existing and new, for trimmed Lasso regularized estimation problems. We include a comparison between the different approaches and an accompanying implementation of the algorithms.