# Fast Sparse Classification for Generalized Linear and Additive Models

arXiv: https://arxiv.org/abs/2202.11389 · HTML: https://mlsanity.com/paper/2202.11389/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 47 references that are in the map.
- First posted on arXiv 23 Feb 2022.

## Builds on (most cited first, 13 of 13)

- [Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges](https://mlsanity.com/paper/2103.11251/) (2021)
- [InterpretML: A Unified Framework for Machine Learning Interpretability](https://mlsanity.com/paper/1909.09223/) (2019)
- [Fast Best Subset Selection: Coordinate Descent and Local Combinatorial Optimization Algorithms](https://mlsanity.com/paper/1803.01454/) (2018)
- [Restricted Strong Convexity Implies Weak Submodularity](https://mlsanity.com/paper/1612.00804/) (2016)
- [Learning Optimized Risk Scores](https://mlsanity.com/paper/1610.00168/) (2016)
- [Learning Sparse Classifiers: Continuous and Mixed Integer Optimization Perspectives](https://mlsanity.com/paper/2001.06471/) (2020)
- [Sparse Classification: a scalable discrete optimization perspective](https://mlsanity.com/paper/1710.01352/) (2017)
- [A Holistic Approach to Interpretability in Financial Lending: Models, Visualizations, and Summary-Explanations](https://mlsanity.com/paper/2106.02605/) (2021)
- [Global and Quadratic Convergence of Newton Hard-Thresholding Pursuit](https://mlsanity.com/paper/1901.02763/) (2019)
- [Sparse Partially Linear Additive Models](https://mlsanity.com/paper/1407.4729/) (2014)
- [Piecewise-Linear Approximation for Feature Subset Selection in a Sequential Logit Model](https://mlsanity.com/paper/1510.05417/) (2015)
- [Iteration complexity analysis of random coordinate descent methods for ℓ0 regularized convex problems](https://mlsanity.com/paper/1403.6622/) (2014)
- [Best-first Search Algorithm for Non-convex Sparse Minimization](https://mlsanity.com/paper/1910.01296/) (2019)

## Built on by (most cited first, 19 of 19)

- [Amazing Things Come From Having Many Good Models](https://mlsanity.com/paper/2407.04846/) (2024)
- [Exploring and Interacting with the Set of Good Sparse Generalized Additive Models](https://mlsanity.com/paper/2303.16047/) (2023)
- [FasterRisk: Fast and Accurate Interpretable Risk Scores](https://mlsanity.com/paper/2210.05846/) (2022)
- [Interpretable Generalized Additive Models for Datasets with Missing Values](https://mlsanity.com/paper/2412.02646/) (2024)
- [Additive Models Explained: A Computational Complexity Approach](https://mlsanity.com/paper/2510.21292/) (2025)
- [LCEN: A Nonlinear, Interpretable Feature Selection and Machine Learning Algorithm](https://mlsanity.com/paper/2402.17120/) (2024)
- [Feature Importance Disparities for Data Bias Investigations](https://mlsanity.com/paper/2303.01704/) (2023)
- [FastSurvival: Hidden Computational Blessings in Training Cox Proportional Hazards Models](https://mlsanity.com/paper/2410.19081/) (2024)
- [Models That Are Interpretable But Not Transparent](https://mlsanity.com/paper/2502.19502/) (2025)
- [Leo Breiman, the Rashomon Effect, and the Occam Dilemma](https://mlsanity.com/paper/2507.03884/) (2025)
- [Provably Explaining Neural Additive Models](https://mlsanity.com/paper/2602.17530/) (2026)
- [When a CBR in Hand is Better than Twins in the Bush](https://mlsanity.com/paper/2305.05111/) (2023)
- [Learning Locally Interpretable Rule Ensemble](https://mlsanity.com/paper/2306.11481/) (2023)
- [FAST: An Optimization Framework for Fast Additive Segmentation in Transparent ML](https://mlsanity.com/paper/2402.12630/) (2024)
- [Prediction Models That Learn to Avoid Missing Values](https://mlsanity.com/paper/2505.03393/) (2025)
- [A Flexible Empirical Bayes Approach to Generalized Linear Models, with Applications to Sparse Logistic Regression](https://mlsanity.com/paper/2601.21217/) (2026)
- [Same Error, Different Function: The Optimizer as an Implicit Prior in Financial Time Series](https://mlsanity.com/paper/2603.02620/) (2026)
- [From Sequential Nodes to GPU Batches: Parallel Branch and Bound for Optimal k-Sparse GLMs](https://mlsanity.com/paper/2605.22188/) (2026)
- [FlagGAM: Rule-Basis Generalized Additive Models for Explainable Tabular Prediction](https://mlsanity.com/paper/2605.31189/) (2026)

## Abstract (arXiv)

We present fast classification techniques for sparse generalized linear and additive models. These techniques can handle thousands of features and thousands of observations in minutes, even in the presence of many highly correlated features. For fast sparse logistic regression, our computational speed-up over other best-subset search techniques owes to linear and quadratic surrogate cuts for the logistic loss that allow us to efficiently screen features for elimination, as well as use of a priority queue that favors a more uniform exploration of features. As an alternative to the logistic loss, we propose the exponential loss, which permits an analytical solution to the line search at each iteration. Our algorithms are generally 2 to 5 times faster than previous approaches. They produce interpretable models that have accuracy comparable to black box models on challenging datasets.