# Gradient Hard Thresholding Pursuit for Sparsity-Constrained Optimization

arXiv: https://arxiv.org/abs/1311.5750 · HTML: https://mlsanity.com/paper/1311.5750/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Nov 2013.

## Built on by (most cited first, 14 of 14)

- [AC/DC: Alternating Compressed/DeCompressed Training of Deep Neural Networks](https://mlsanity.com/paper/2106.12379/) (2021)
- [Stability and Risk Bounds of Iterative Hard Thresholding](https://mlsanity.com/paper/2203.09413/) (2022)
- [Deep Learning of Dynamic Systems using System Identification Toolbox(TM)](https://mlsanity.com/paper/2409.07642/) (2024)
- [Zero-shot Concept Bottleneck Models](https://mlsanity.com/paper/2502.09018/) (2025)
- [Resource Constrained Model Compression via Minimax Optimization for Spiking Neural Networks](https://mlsanity.com/paper/2308.04672/) (2023)
- [Approximate Frank-Wolfe Algorithms over Graph-structured Support Sets](https://mlsanity.com/paper/2107.00472/) (2021)
- [Unified Signal Compression Using a GAN with Iterative Latent Representation Optimization](https://mlsanity.com/paper/2109.11168/) (2021)
- [A Data-Driven Line Search Rule for Support Recovery in High-dimensional Data Analysis](https://mlsanity.com/paper/2111.10806/) (2021)
- [Best Subset Selection with Efficient Primal-Dual Algorithm](https://mlsanity.com/paper/2207.02058/) (2022)
- [Block-Structured Optimization for Subgraph Detection in Interdependent Networks](https://mlsanity.com/paper/2210.02702/) (2022)
- [Learning Large Scale Sparse Models](https://mlsanity.com/paper/2301.10958/) (2023)
- [Dynamic Incremental Optimization for Best Subset Selection](https://mlsanity.com/paper/2402.02322/) (2024)
- [Stochastic Variance-Reduced Iterative Hard Thresholding in Graph Sparsity Optimization](https://mlsanity.com/paper/2407.16968/) (2024)
- [Adaptive Threshold-Driven Continuous Greedy Method for Scalable Submodular Optimization](https://mlsanity.com/paper/2604.03419/) (2026)

## Abstract (arXiv)

Hard Thresholding Pursuit (HTP) is an iterative greedy selection procedure for finding sparse solutions of underdetermined linear systems. This method has been shown to have strong theoretical guarantee and impressive numerical performance. In this paper, we generalize HTP from compressive sensing to a generic problem setup of sparsity-constrained convex optimization. The proposed algorithm iterates between a standard gradient descent step and a hard thresholding step with or without debiasing. We prove that our method enjoys the strong guarantees analogous to HTP in terms of rate of convergence and parameter estimation accuracy. Numerical evidences show that our method is superior to the state-of-the-art greedy selection methods in sparse logistic regression and sparse precision matrix estimation tasks.