# Escoin: Efficient Sparse Convolutional Neural Network Inference on GPUs

arXiv: https://arxiv.org/abs/1802.10280 · HTML: https://mlsanity.com/paper/1802.10280/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Feb 2018.

## Built on by (most cited first, 11 of 11)

- [Only Train Once: A One-Shot Neural Network Training And Pruning Framework](https://mlsanity.com/paper/2107.07467/) (2021)
- [FIARSE: Model-Heterogeneous Federated Learning via Importance-Aware Submodel Extraction](https://mlsanity.com/paper/2407.19389/) (2024)
- [RED++ : Data-Free Pruning of Deep Neural Networks via Input Splitting and Output Merging](https://mlsanity.com/paper/2110.01397/) (2021)
- [SparseDNN: Fast Sparse Deep Learning Inference on CPUs](https://mlsanity.com/paper/2101.07948/) (2021)
- [High Performance Convolution Using Sparsity and Patterns for Inference in Deep Convolutional Neural Networks](https://mlsanity.com/paper/2104.08314/) (2021)
- [Speedup deep learning models on GPU by taking advantage of efficient unstructured pruning and bit-width reduction](https://mlsanity.com/paper/2112.15445/) (2021)
- [FSCNN: A Fast Sparse Convolution Neural Network Inference System](https://mlsanity.com/paper/2212.08815/) (2022)
- [On the Compression of Natural Language Models](https://mlsanity.com/paper/2112.11480/) (2021)
- [STen: Productive and Efficient Sparsity in PyTorch](https://mlsanity.com/paper/2304.07613/) (2023)
- [Adaptive Structured Pruning of Convolutional Neural Networks for Time Series Classification](https://mlsanity.com/paper/2602.12744/) (2026)
- [Efficient Human-Contact Representation for Human-Scene Interaction](https://mlsanity.com/paper/2608.09388/) (2026)

## Abstract (arXiv)

Deep neural networks have achieved remarkable accuracy in many artificial intelligence applications, e.g. computer vision, at the cost of a large number of parameters and high computational complexity. Weight pruning can compress DNN models by removing redundant parameters in the networks, but it brings sparsity in the weight matrix, and therefore makes the computation inefficient on GPUs. Although pruning can remove more than 80% of the weights, it actually hurts inference performance (speed) when running models on GPUs. Two major problems cause this unsatisfactory performance on GPUs. First, lowering convolution onto matrix multiplication reduces data reuse opportunities and wastes memory bandwidth. Second, the sparsity brought by pruning makes the computation irregular, which leads to inefficiency when running on massively parallel GPUs. To overcome these two limitations, we propose Escort, an efficient sparse convolutional neural networks on GPUs. Instead of using the lowering method, we choose to compute the sparse convolutions directly. We then orchestrate the parallelism and locality for the direct sparse convolution kernel, and apply customized optimization techniques to further improve performance. Evaluation on NVIDIA GPUs show that Escort can improve sparse convolution speed by 2.63x and 3.07x, and inference speed by 1.43x and 1.69x, compared to CUBLAS and CUSPARSE respectively.