# Explicit Group Sparse Projection with Applications to Deep Learning and NMF

arXiv: https://arxiv.org/abs/1912.03896 · HTML: https://mlsanity.com/paper/1912.03896/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Dec 2019.

## Built on by (most cited first, 6 of 6)

- [Fair Wasserstein Coresets](https://mlsanity.com/paper/2311.05436/) (2023)
- [A consistent and flexible framework for deep matrix factorizations](https://mlsanity.com/paper/2206.10693/) (2022)
- [Sparse-Reg: Improving Sample Complexity in Offline Reinforcement Learning using Sparsity](https://mlsanity.com/paper/2506.17155/) (2025)
- [SalientGrads: Sparse Models for Communication Efficient and Data Aware Distributed Federated Training](https://mlsanity.com/paper/2304.07488/) (2023)
- [SSFL: Discovering Sparse Unified Subnetworks at Initialization for Efficient Federated Learning](https://mlsanity.com/paper/2405.09037/) (2024)
- [Identification of NMF by choosing maximum-volume basis vectors](https://mlsanity.com/paper/2603.24227/) (2026)

## Abstract (arXiv)

We design a new sparse projection method for a set of vectors that guarantees a desired average sparsity level measured leveraging the popular Hoyer measure (an affine function of the ratio of the ℓ1 and ℓ2 norms). Existing approaches either project each vector individually or require the use of a regularization parameter which implicitly maps to the average ℓ0-measure of sparsity. Instead, in our approach we set the sparsity level for the whole set explicitly and simultaneously project a group of vectors with the sparsity level of each vector tuned automatically. We show that the computational complexity of our projection operator is linear in the size of the problem. Additionally, we propose a generalization of this projection by replacing the ℓ1 norm by its weighted version. We showcase the efficacy of our approach in both supervised and unsupervised learning tasks on image datasets including CIFAR10 and ImageNet. In deep neural network pruning, the sparse models produced by our method on ResNet50 have significantly higher accuracies at corresponding sparsity values compared to existing competitors. In nonnegative matrix factorization, our approach yields competitive reconstruction errors against state-of-the-art algorithms.