# Enabling Sparse Winograd Convolution by Native Pruning

arXiv: https://arxiv.org/abs/1702.08597 · HTML: https://mlsanity.com/paper/1702.08597/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Feb 2017.

## Built on by (most cited first, 6 of 6)

- [Searching for Fast Model Families on Datacenter Accelerators](https://mlsanity.com/paper/2102.05610/) (2021)
- [Impact of Parameter Sparsity on Stochastic Gradient MCMC Methods for Bayesian Deep Learning](https://mlsanity.com/paper/2202.03770/) (2022)
- [Going Further With Winograd Convolutions: Tap-Wise Quantization for Efficient Inference on 4x4 Tile](https://mlsanity.com/paper/2209.12982/) (2022)
- [Low-Rank Winograd Transformation for 3D Convolutional Neural Networks](https://mlsanity.com/paper/2301.11180/) (2023)
- [STen: Productive and Efficient Sparsity in PyTorch](https://mlsanity.com/paper/2304.07613/) (2023)
- [Fast Convolution based on Winograd Minimum Filtering: Introduction and Development](https://mlsanity.com/paper/2111.00977/) (2021)

## Abstract (arXiv)

Sparse methods and the use of Winograd convolutions are two orthogonal approaches, each of which significantly accelerates convolution computations in modern CNNs. Sparse Winograd merges these two and thus has the potential to offer a combined performance benefit. Nevertheless, training convolution layers so that the resulting Winograd kernels are sparse has not hitherto been very successful. By introducing a Winograd layer in place of a standard convolution layer, we can learn and prune Winograd coefficients "natively" and obtain sparsity level beyond 90% with only 0.1% accuracy loss with AlexNet on ImageNet dataset. Furthermore, we present a sparse Winograd convolution algorithm and implementation that exploits the sparsity, achieving up to 31.7 effective TFLOP/s in 32-bit precision on a latest Intel Xeon CPU, which corresponds to a 5.4x speedup over a state-of-the-art dense convolution implementation.