# Faster CNNs with Direct Sparse Convolutions and Guided Pruning

arXiv: https://arxiv.org/abs/1608.01409 · HTML: https://mlsanity.com/paper/1608.01409/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Aug 2016.

## Built on by (most cited first, 30 of 39)

- [Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications](https://mlsanity.com/paper/2204.11786/) (2022)
- [SepPrune: Structured Pruning for Efficient Deep Speech Separation](https://mlsanity.com/paper/2505.12079/) (2025)
- [DECORE: Deep Compression with Reinforcement Learning](https://mlsanity.com/paper/2106.06091/) (2021)
- [Coarsening the Granularity: Towards Structurally Sparse Lottery Tickets](https://mlsanity.com/paper/2202.04736/) (2022)
- [SGLP: A Similarity Guided Fast Layer Partition Pruning for Compressing Large Deep Models](https://mlsanity.com/paper/2410.14720/) (2024)
- [A Generic Layer Pruning Method for Signal Modulation Recognition Deep Learning Models](https://mlsanity.com/paper/2406.07929/) (2024)
- [SparseDNN: Fast Sparse Deep Learning Inference on CPUs](https://mlsanity.com/paper/2101.07948/) (2021)
- [SparseProp: Efficient Sparse Backpropagation for Faster Training of Neural Networks](https://mlsanity.com/paper/2302.04852/) (2023)
- [Dimensionality Reduced Training by Pruning and Freezing Parts of a Deep Neural Network, a Survey](https://mlsanity.com/paper/2205.08099/) (2022)
- [Sparse Teachers Can Be Dense with Knowledge](https://mlsanity.com/paper/2210.03923/) (2022)
- [SONIC: A Sparse Neural Network Inference Accelerator with Silicon Photonics for Energy-Efficient Deep Learning](https://mlsanity.com/paper/2109.04459/) (2021)
- [Are Straight-Through gradients and Soft-Thresholding all you need for Sparse Training?](https://mlsanity.com/paper/2212.01076/) (2022)
- [An Effective Information Theoretic Framework for Channel Pruning](https://mlsanity.com/paper/2408.16772/) (2024)
- [AdaptCL: Efficient Collaborative Learning with Dynamic and Adaptive Pruning](https://mlsanity.com/paper/2106.14126/) (2021)
- [Doge Tickets: Uncovering Domain-general Language Models by Playing Lottery Tickets](https://mlsanity.com/paper/2207.09638/) (2022)
- [DLSIA: Deep Learning for Scientific Image Analysis](https://mlsanity.com/paper/2308.02559/) (2023)
- [FCOS: A Two-Stage Recoverable Model Pruning Framework for Automatic Modulation Recognition](https://mlsanity.com/paper/2505.21571/) (2025)
- [Visual Detector Compression via Location-Aware Discriminant Analysis](https://mlsanity.com/paper/2509.17968/) (2025)
- [High Performance Convolution Using Sparsity and Patterns for Inference in Deep Convolutional Neural Networks](https://mlsanity.com/paper/2104.08314/) (2021)
- [Search Spaces for Neural Model Training](https://mlsanity.com/paper/2105.12920/) (2021)
- [Speedup deep learning models on GPU by taking advantage of efficient unstructured pruning and bit-width reduction](https://mlsanity.com/paper/2112.15445/) (2021)
- [Impact of Parameter Sparsity on Stochastic Gradient MCMC Methods for Bayesian Deep Learning](https://mlsanity.com/paper/2202.03770/) (2022)
- [RNAS-CL: Robust Neural Architecture Search by Cross-Layer Knowledge Distillation](https://mlsanity.com/paper/2301.08092/) (2023)
- [Visual Saliency-Guided Channel Pruning for Deep Visual Detectors in Autonomous Driving](https://mlsanity.com/paper/2303.02512/) (2023)
- [Filter Pruning for Efficient CNNs via Knowledge-driven Differential Filter Sampler](https://mlsanity.com/paper/2307.00198/) (2023)
- [Pruning-based Data Selection and Network Fusion for Efficient Deep Learning](https://mlsanity.com/paper/2501.01118/) (2025)
- [Lightweight Software Kernels and Hardware Extensions for Efficient Sparse Deep Neural Networks on Microcontrollers](https://mlsanity.com/paper/2503.06183/) (2025)
- [Hands-on Guidance for Distilling Object Detectors](https://mlsanity.com/paper/2103.14337/) (2021)
- [cuConv: A CUDA Implementation of Convolution for CNN Inference](https://mlsanity.com/paper/2103.16234/) (2021)
- [STen: Productive and Efficient Sparsity in PyTorch](https://mlsanity.com/paper/2304.07613/) (2023)

## Abstract (arXiv)

Phenomenally successful in practical inference problems, convolutional neural networks (CNN) are widely deployed in mobile devices, data centers, and even supercomputers. The number of parameters needed in CNNs, however, are often large and undesirable. Consequently, various methods have been developed to prune a CNN once it is trained. Nevertheless, the resulting CNNs offer limited benefits. While pruning the fully connected layers reduces a CNN's size considerably, it does not improve inference speed noticeably as the compute heavy parts lie in convolutions. Pruning CNNs in a way that increase inference speed often imposes specific sparsity structures, thus limiting the achievable sparsity levels. We present a method to realize simultaneously size economy and speed improvement while pruning CNNs. Paramount to our success is an efficient general sparse-with-dense matrix multiplication implementation that is applicable to convolution of feature maps with kernels of arbitrary sparsity patterns. Complementing this, we developed a performance model that predicts sweet spots of sparsity levels for different layers and on different computer architectures. Together, these two allow us to demonstrate 3.1--7.3× convolution speedups over dense convolution in AlexNet, on Intel Atom, Xeon, and Xeon Phi processors, spanning the spectrum from mobile devices to supercomputers. We also open source our project at https://github.com/IntelLabs/SkimCaffe.