# Nonconvex Sparse Learning via Stochastic Optimization with Progressive Variance Reduction

arXiv: https://arxiv.org/abs/1605.02711 · HTML: https://mlsanity.com/paper/1605.02711/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 May 2016.

## Built on by (most cited first, 6 of 6)

- [Distributed Learning Systems with First-order Methods](https://mlsanity.com/paper/2104.05245/) (2021)
- [Sparse Polyak: an adaptive step size rule for high-dimensional M-estimation](https://mlsanity.com/paper/2509.09802/) (2025)
- [SADAM: Stochastic Adam, A Stochastic Operator for First-Order Gradient-based Optimizer](https://mlsanity.com/paper/2205.10247/) (2022)
- [Stochastic Variance-Reduced Iterative Hard Thresholding in Graph Sparsity Optimization](https://mlsanity.com/paper/2407.16968/) (2024)
- [Sparse Polyak with optimal thresholding operators for high-dimensional M-estimation](https://mlsanity.com/paper/2511.18167/) (2025)
- [New Insight of Variance reduce in Zero-Order Hard-Thresholding: Mitigating Gradient Error and Expansivity Contradictions](https://mlsanity.com/paper/2605.18035/) (2026)

## Abstract (arXiv)

We propose a stochastic variance reduced optimization algorithm for solving sparse learning problems with cardinality constraints. Sufficient conditions are provided, under which the proposed algorithm enjoys strong linear convergence guarantees and optimal estimation accuracy in high dimensions. We further extend the proposed algorithm to an asynchronous parallel variant with a near linear speedup. Numerical experiments demonstrate the efficiency of our algorithm in terms of both parameter estimation and computational performance.