# Beyond L1: Faster and Better Sparse Models with skglm

arXiv: https://arxiv.org/abs/2204.07826 · HTML: https://mlsanity.com/paper/2204.07826/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 87 references that are in the map.
- First posted on arXiv 16 Apr 2022.

## Builds on (most cited first, 27 of 27)

- [Stochastic model-based minimization of weakly convex functions](https://mlsanity.com/paper/1803.06523/) (2018)
- [Model Selection in Bayesian Neural Networks via Horseshoe Priors](https://mlsanity.com/paper/1705.10388/) (2017)
- [Coordinate Descent Converges Faster with the Gauss-Southwell Rule Than Random Selection](https://mlsanity.com/paper/1506.00552/) (2015)
- [Gap Safe screening rules for sparsity enforcing penalties](https://mlsanity.com/paper/1611.05780/) (2016)
- [Implicit differentiation of Lasso-type models for hyperparameter optimization](https://mlsanity.com/paper/2002.08943/) (2020)
- [Mind the duality gap: safer rules for the Lasso](https://mlsanity.com/paper/1505.03410/) (2015)
- [Celer: a Fast Solver for the Lasso with Dual Extrapolation](https://mlsanity.com/paper/1802.07481/) (2018)
- [Accelerated, Parallel and Proximal Coordinate Descent](https://mlsanity.com/paper/1312.5799/) (2013)
- [A Survey on Nonconvex Regularization Based Sparse and Low-Rank Recovery in Signal Processing, Statistics, and Machine Learning](https://mlsanity.com/paper/1808.05403/) (2018)
- [Regularized Nonlinear Acceleration](https://mlsanity.com/paper/1606.04133/) (2016)
- [Dynamic Screening: Accelerating First-Order Algorithms for the Lasso and Group-Lasso](https://mlsanity.com/paper/1412.4080/) (2014)
- [Screening Rules for Lasso with Non-Convex Sparse Regularizers](https://mlsanity.com/paper/1902.06125/) (2019)
- [Anderson Acceleration of Proximal Gradient Methods](https://mlsanity.com/paper/1910.08590/) (2019)
- [Screening Rules and its Complexity for Active Set Identification](https://mlsanity.com/paper/2009.02709/) (2020)
- [Stochastic Anderson Mixing for Nonconvex Stochastic Optimization](https://mlsanity.com/paper/2110.01543/) (2021)
- [Trajectory of Alternating Direction Method of Multipliers and Adaptive Acceleration](https://mlsanity.com/paper/1906.10114/) (2019)
- [Anderson Acceleration for Nonconvex ADMM Based on Douglas-Rachford Splitting](https://mlsanity.com/paper/2006.14539/) (2020)
- [Anderson acceleration of coordinate descent](https://mlsanity.com/paper/2011.10065/) (2020)
- [Picasso: A Sparse Learning Library for High Dimensional Data Analysis in R and Python](https://mlsanity.com/paper/2006.15261/) (2020)
- [Dual Extrapolation for Sparse Generalized Linear Models](https://mlsanity.com/paper/1907.05830/) (2019)
- [Model identification and local linear convergence of coordinate descent](https://mlsanity.com/paper/2010.11825/) (2020)
- [Low Complexity Regularization of Linear Inverse Problems](https://mlsanity.com/paper/1407.1598/) (2014)
- [A Multi-step Inertial Forward--Backward Splitting Method for Non-convex Optimization](https://mlsanity.com/paper/1606.02118/) (2016)
- [WHInter: A Working set algorithm for High-dimensional sparse second order Interaction models](https://mlsanity.com/paper/1802.05980/) (2018)
- [Efficiency of Coordinate Descent Methods For Structured Nonconvex Optimization](https://mlsanity.com/paper/1909.00918/) (2019)
- [Continuation Path with Linear Convergence Rate](https://mlsanity.com/paper/2112.05104/) (2021)
- [The iterative reweighted Mixed-Norm Estimate for spatio-temporal MEG/EEG source reconstruction](https://mlsanity.com/paper/1607.08458/) (2016)

## Built on by (most cited first, 16 of 16)

- [Benchopt: Reproducible, efficient and collaborative optimization benchmarks](https://mlsanity.com/paper/2206.13424/) (2022)
- [Conformalization of Sparse Generalized Linear Models](https://mlsanity.com/paper/2307.05109/) (2023)
- [Anderson Acceleration For Bioinformatics-Based Machine Learning](https://mlsanity.com/paper/2302.00347/) (2023)
- [FedECA: Federated External Control Arms for Causal Inference with Time-To-Event Data in Distributed Settings](https://mlsanity.com/paper/2311.16984/) (2023)
- [Integrated path stability selection](https://mlsanity.com/paper/2403.15877/) (2024)
- [The Choice of Normalization Influences Shrinkage in Regularized Regression](https://mlsanity.com/paper/2501.03821/) (2025)
- [FastSurvival: Hidden Computational Blessings in Training Cox Proportional Hazards Models](https://mlsanity.com/paper/2410.19081/) (2024)
- [Scalable Approximate Algorithms for Optimal Transport Linear Models](https://mlsanity.com/paper/2504.04609/) (2025)
- [A Generic Branch-and-Bound Algorithm for ℓ0-Penalized Problems with Supplementary Material](https://mlsanity.com/paper/2506.03974/) (2025)
- [El0ps: An Exact L0-regularized Problems Solver](https://mlsanity.com/paper/2506.06373/) (2025)
- [Proximal Operators of Sorted Nonconvex Penalties](https://mlsanity.com/paper/2506.15315/) (2025)
- [SigBERT: Combining Narrative Medical Reports and Rough Path Signature Theory for Survival Risk Estimation in Oncology](https://mlsanity.com/paper/2507.22941/) (2025)
- [Efficient Solvers for SLOPE in R, Python, Julia, and C++](https://mlsanity.com/paper/2511.02430/) (2025)
- [A Flexible Empirical Bayes Approach to Generalized Linear Models, with Applications to Sparse Logistic Regression](https://mlsanity.com/paper/2601.21217/) (2026)
- [A block-coordinate descent framework for non-convex composite optimization. Application to sparse precision matrix estimation](https://mlsanity.com/paper/2601.21467/) (2026)
- [Simpler Methods Work Better for L1 Penalized Logistic Models and Large Datasets](https://mlsanity.com/paper/2609.23995/) (2026)

## Abstract (arXiv)

We propose a new fast algorithm to estimate any sparse generalized linear model with convex or non-convex separable penalties. Our algorithm is able to solve problems with millions of samples and features in seconds, by relying on coordinate descent, working sets and Anderson acceleration. It handles previously unaddressed models, and is extensively shown to improve state-of-art algorithms. We provide a flexible, scikit-learn compatible package, which easily handles customized datafits and penalties.