# A Risk Ratio Comparison of l0 and l1 Penalized Regression

arXiv: https://arxiv.org/abs/1510.06319 · HTML: https://mlsanity.com/paper/1510.06319/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Oct 2015.

## Built on by (most cited first, 2 of 2)

- [Modelling COVID-19 Pandemic Dynamics Using Transparent, Interpretable, Parsimonious and Simulatable (TIPS) Machine Learning Models: A Case Study from Systems Thinking and System Identification Perspectives](https://mlsanity.com/paper/2111.01763/) (2021)
- [Probabilistic and nonlinear compressive sensing](https://mlsanity.com/paper/2509.15060/) (2025)

## Abstract (arXiv)

There has been an explosion of interest in using l1-regularization in place of l0-regularization for feature selection. We present theoretical results showing that while l1-penalized linear regression never outperforms l0-regularization by more than a constant factor, in some cases using an l1 penalty is infinitely worse than using an l0 penalty. We also show that the "optimal" l1 solutions are often inferior to l0 solutions found using stepwise regression. We also compare algorithms for solving these two problems and show that although solutions can be found efficiently for the l1 problem, the "optimal" l1 solutions are often inferior to l0 solutions found using greedy classic stepwise regression. Furthermore, we show that solutions obtained by solving the convex l1 problem can be improved by selecting the best of the l1 models (for different regularization penalties) by using an l0 criterion. In other words, an approximate solution to the right problem can be better than the exact solution to the wrong problem.