# Parameterized Structured Pruning for Deep Neural Networks

arXiv: https://arxiv.org/abs/1906.05180 · HTML: https://mlsanity.com/paper/1906.05180/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jun 2019.

## Built on by (most cited first, 4 of 4)

- [DepGraph: Towards Any Structural Pruning](https://mlsanity.com/paper/2301.12900/) (2023)
- [Compressing the Backward Pass of Large-Scale Neural Architectures by Structured Activation Pruning](https://mlsanity.com/paper/2311.16883/) (2023)
- [Less Memory Means smaller GPUs: Backpropagation with Compressed Activations](https://mlsanity.com/paper/2409.11902/) (2024)
- [PruneForget: Joint Unlearning and Pruning of Vision Models](https://mlsanity.com/paper/2609.32162/) (2026)

## Abstract (arXiv)

As a result of the growing size of Deep Neural Networks (DNNs), the gap to hardware capabilities in terms of memory and compute increases. To effectively compress DNNs, quantization and connection pruning are usually considered. However, unconstrained pruning usually leads to unstructured parallelism, which maps poorly to massively parallel processors, and substantially reduces the efficiency of general-purpose processors. Similar applies to quantization, which often requires dedicated hardware. We propose Parameterized Structured Pruning (PSP), a novel method to dynamically learn the shape of DNNs through structured sparsity. PSP parameterizes structures (e.g. channel- or layer-wise) in a weight tensor and leverages weight decay to learn a clear distinction between important and unimportant structures. As a result, PSP maintains prediction performance, creates a substantial amount of sparsity that is structured and, thus, easy and efficient to map to a variety of massively parallel processors, which are mandatory for utmost compute power and energy efficiency. PSP is experimentally validated on the popular CIFAR10/100 and ILSVRC2012 datasets using ResNet and DenseNet architectures, respectively.