# CUP: Cluster Pruning for Compressing Deep Neural Networks

arXiv: https://arxiv.org/abs/1911.08630 · HTML: https://mlsanity.com/paper/1911.08630/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Nov 2019.

## Built on by (most cited first, 7 of 7)

- [Convolutional Neural Network Pruning with Structural Redundancy Reduction](https://mlsanity.com/paper/2104.03438/) (2021)
- [NeuroCartography: Scalable Automatic Visual Summarization of Concepts in Deep Neural Networks](https://mlsanity.com/paper/2108.12931/) (2021)
- [VisCUIT: Visual Auditor for Bias in CNN Image Classifier](https://mlsanity.com/paper/2204.05899/) (2022)
- [REPrune: Channel Pruning via Kernel Representative Selection](https://mlsanity.com/paper/2402.17862/) (2024)
- [TropNNC: Structured Neural Network Compression Using Tropical Geometry](https://mlsanity.com/paper/2409.03945/) (2024)
- [Towards Regression-Free Neural Networks for Diverse Compute Platforms](https://mlsanity.com/paper/2209.13740/) (2022)
- [Ef-QuantFace: Streamlined Face Recognition with Small Data and Low-Bit Precision](https://mlsanity.com/paper/2402.18163/) (2024)

## Abstract (arXiv)

We propose Cluster Pruning (CUP) for compressing and accelerating deep neural networks. Our approach prunes similar filters by clustering them based on features derived from both the incoming and outgoing weight connections. With CUP, we overcome two limitations of prior work-(1) non-uniform pruning: CUP can efficiently determine the ideal number of filters to prune in each layer of a neural network. This is in contrast to prior methods that either prune all layers uniformly or otherwise use resource-intensive methods such as manual sensitivity analysis or reinforcement learning to determine the ideal number. (2) Single-shot operation: We extend CUP to CUP-SS (for CUP single shot) whereby pruning is integrated into the initial training phase itself. This leads to large savings in training time compared to traditional pruning pipelines. Through extensive evaluation on multiple datasets (MNIST, CIFAR-10, and Imagenet) and models(VGG-16, Resnets-18/34/56) we show that CUP outperforms recent state of the art. Specifically, CUP-SS achieves 2.2x flops reduction for a Resnet-50 model trained on Imagenet while staying within 0.9% top-5 accuracy. It saves over 14 hours in training time with respect to the original Resnet-50. The code to reproduce results is available.