# A Reduced-Space Algorithm for Minimizing ℓ1-Regularized Convex Functions

arXiv: https://arxiv.org/abs/1602.07018 · HTML: https://mlsanity.com/paper/1602.07018/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Feb 2016.

## Built on by (most cited first, 7 of 7)

- [OTOV2: Automatic, Generic, User-Friendly](https://mlsanity.com/paper/2303.06862/) (2023)
- [Automatic Joint Structured Pruning and Quantization for Efficient Neural Network Training and Compression](https://mlsanity.com/paper/2502.16638/) (2025)
- [OTOv3: Automatic Architecture-Agnostic Neural Network Training and Compression from Structured Pruning to Erasing Operators](https://mlsanity.com/paper/2312.09411/) (2023)
- [HESSO: Towards Automatic Efficient and User Friendly Any Neural Network Training and Pruning](https://mlsanity.com/paper/2409.09085/) (2024)
- [WINA: Weight Informed Neuron Activation for Accelerating Large Language Model Inference](https://mlsanity.com/paper/2505.19427/) (2025)
- [FSCNN: A Fast Sparse Convolution Neural Network Inference System](https://mlsanity.com/paper/2212.08815/) (2022)
- [Alternating Iteratively Reweighted ℓ1 and Subspace Newton Algorithms for Nonconvex Sparse Optimization](https://mlsanity.com/paper/2407.17216/) (2024)

## Abstract (arXiv)

We present a new method for minimizing the sum of a differentiable convex function and an ℓ1-norm regularizer. The main features of the new method include: (i) an evolving set of indices corresponding to variables that are predicted to be nonzero at a solution (i.e., the support); (ii) a reduced-space subproblem defined in terms of the predicted support; (iii) conditions that determine how accurately each subproblem must be solved, which allow for Newton, Newton-CG, and coordinate-descent techniques to be employed; (iv) a computationally practical condition that determines when the predicted support should be updated; and (v) a reduced proximal gradient step that ensures sufficient decrease in the objective function when it is decided that variables should be added to the predicted support. We prove a convergence guarantee for our method and demonstrate its efficiency on a large set of model prediction problems.