# Simpler Methods Work Better for L1 Penalized Logistic Models and Large Datasets

arXiv: https://arxiv.org/abs/2609.23995 · HTML: https://mlsanity.com/paper/2609.23995/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 36 of its 95 references that are in the map.
- First posted on arXiv 21 Sep 2026.

## Builds on (most cited first, 30 of 36)

- [SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives](https://mlsanity.com/paper/1407.0202/) (2014)
- [A Metric Learning Reality Check](https://mlsanity.com/paper/2003.08505/) (2020)
- [A Step Toward Quantifying Independently Reproducible Machine Learning Research](https://mlsanity.com/paper/1909.06674/) (2019)
- [SOREL-20M: A Large Scale Benchmark Dataset for Malicious PE Detection](https://mlsanity.com/paper/2012.07634/) (2020)
- [Gap Safe screening rules for sparsity enforcing penalties](https://mlsanity.com/paper/1611.05780/) (2016)
- [Celer: a Fast Solver for the Lasso with Dual Extrapolation](https://mlsanity.com/paper/1802.07481/) (2018)
- [Benchopt: Reproducible, efficient and collaborative optimization benchmarks](https://mlsanity.com/paper/2206.13424/) (2022)
- [EMBER2024 -- A Benchmark Dataset for Holistic Evaluation of Malware Classifiers](https://mlsanity.com/paper/2506.05074/) (2025)
- [ASAGA: Asynchronous Parallel SAGA](https://mlsanity.com/paper/1606.04809/) (2016)
- [Automatic Yara Rule Generation Using Biclustering](https://mlsanity.com/paper/2009.03779/) (2020)
- [MOTIF: A Large Malware Reference Dataset with Ground Truth Family Labels](https://mlsanity.com/paper/2111.15031/) (2021)
- [spred: Solving L1 Penalty with SGD](https://mlsanity.com/paper/2210.01212/) (2022)
- [Research Reproducibility as a Survival Analysis](https://mlsanity.com/paper/2012.09932/) (2020)
- [Beyond L1: Faster and Better Sparse Models with skglm](https://mlsanity.com/paper/2204.07826/) (2022)
- [A Safe Screening Rule for Sparse Logistic Regression](https://mlsanity.com/paper/1307.4145/) (2013)
- [Screening Rules for Lasso with Non-Convex Sparse Regularizers](https://mlsanity.com/paper/1902.06125/) (2019)
- [Reproducibility in Multiple Instance Learning: A Case For Algorithmic Unit Tests](https://mlsanity.com/paper/2310.17867/) (2023)
- [KiloGrams: Very Large N-Grams for Malware Classification](https://mlsanity.com/paper/1908.00200/) (2019)
- [What Do Machine Learning Researchers Mean by "Reproducible"?](https://mlsanity.com/paper/2412.03854/) (2024)
- [Does the Market of Citations Reward Reproducible Work?](https://mlsanity.com/paper/2204.03829/) (2022)
- [A Siren Song of Open Source Reproducibility](https://mlsanity.com/paper/2204.04372/) (2022)
- [An L-BFGS-B approach for linear and nonlinear system identification under ℓ1 and group-Lasso regularization](https://mlsanity.com/paper/2403.03827/) (2024)
- [Expanding boundaries of Gap Safe screening](https://mlsanity.com/paper/2102.10846/) (2021)
- [A unified sparse matrix data format for efficient general sparse matrix-vector multiply on modern processors with wide SIMD units](https://mlsanity.com/paper/1307.6209/) (2013)
- [The biglasso Package: A Memory- and Computation-Efficient Solver for Lasso Model Fitting with Big Data in R](https://mlsanity.com/paper/1701.05936/) (2017)
- [Anderson acceleration of coordinate descent](https://mlsanity.com/paper/2011.10065/) (2020)
- [L1-Regularized Distributed Optimization: A Communication-Efficient Primal-Dual Framework](https://mlsanity.com/paper/1512.04011/) (2015)
- [Look-Ahead Screening Rules for the Lasso](https://mlsanity.com/paper/2105.05648/) (2021)
- [Examining the Effect of Implementation Factors on Deep Learning Reproducibility](https://mlsanity.com/paper/2312.06633/) (2023)
- [High-Dimensional Distributed Sparse Classification with Scalable Communication-Efficient Global Updates](https://mlsanity.com/paper/2407.06346/) (2024)

## Abstract (arXiv)

Linear models with an L1-norm penalty remain state-of-the-art for high-dimensional (d > 1,000,000) tasks, offering a straightforward method for solving real-world industry problems. Despite their widespread use in industry and utility, many L1 solvers are not effective for general use, are prohibitively slow, and are ineffective in parallelization. This makes them difficult to train in an MLOps pipeline on large industry-scale corpora. In this work, we test several proposed ``state-of-the-art'' solutions from the literature and find that older methods are currently far superior for general use. We also identify several recommendations for academics to perform research that avoids erroneously overconfident results, which can prevent the transition to production use. Equally surprising, we find that a new and simple baseline, using LBFGS on a sub-gradient, is highly effective with minor tweaks, despite being dismissed in the literature for theoretical non-convergence. In practice, we find it is an easier-to-support and easier-to-scale method for production use.