# Efficient Sparse-Winograd Convolutional Neural Networks

arXiv: https://arxiv.org/abs/1802.06367 · HTML: https://mlsanity.com/paper/1802.06367/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Feb 2018.

## Built on by (most cited first, 20 of 20)

- [EfficientFormer: Vision Transformers at MobileNet Speed](https://mlsanity.com/paper/2206.01191/) (2022)
- [Convolutional Neural Network Pruning with Structural Redundancy Reduction](https://mlsanity.com/paper/2104.03438/) (2021)
- [Real-Time Neural Light Field on Mobile Devices](https://mlsanity.com/paper/2212.08057/) (2022)
- [Dual-side Sparse Tensor Core](https://mlsanity.com/paper/2105.09564/) (2021)
- [BiViT: Extremely Compressed Binary Vision Transformer](https://mlsanity.com/paper/2211.07091/) (2022)
- [Searching for Fast Model Families on Datacenter Accelerators](https://mlsanity.com/paper/2102.05610/) (2021)
- [Interspace Pruning: Using Adaptive Filter Representations to Improve Training of Sparse CNNs](https://mlsanity.com/paper/2203.07808/) (2022)
- [Accelerating Text-to-Image Editing via Cache-Enabled Sparse Diffusion Inference](https://mlsanity.com/paper/2305.17423/) (2023)
- [Energy-Latency Attacks to On-Device Neural Networks via Sponge Poisoning](https://mlsanity.com/paper/2305.03888/) (2023)
- [Winograd Convolution: A Perspective from Fault Tolerance](https://mlsanity.com/paper/2202.08675/) (2022)
- [cuConv: A CUDA Implementation of Convolution for CNN Inference](https://mlsanity.com/paper/2103.16234/) (2021)
- [EcoFlow: Efficient Convolutional Dataflows for Low-Power Neural Network Accelerators](https://mlsanity.com/paper/2202.02310/) (2022)
- [Estimating the Power Consumption of Heterogeneous Devices when performing AI Inference](https://mlsanity.com/paper/2207.06150/) (2022)
- [Low-Rank Winograd Transformation for 3D Convolutional Neural Networks](https://mlsanity.com/paper/2301.11180/) (2023)
- [Exploring Winograd Convolution for Cost-effective Neural Network Fault Tolerance](https://mlsanity.com/paper/2308.08230/) (2023)
- [YFlows: Systematic Dataflow Exploration and Code Generation for Efficient Neural Network Inference using SIMD Architectures on CPUs](https://mlsanity.com/paper/2310.00574/) (2023)
- [Fast Convolution based on Winograd Minimum Filtering: Introduction and Development](https://mlsanity.com/paper/2111.00977/) (2021)
- [Low-Energy Convolutional Neural Networks (CNNs) using Hadamard Method](https://mlsanity.com/paper/2209.09106/) (2022)
- [Fast-ParC: Capturing Position Aware Global Feature for ConvNets and ViTs](https://mlsanity.com/paper/2210.04020/) (2022)
- [NOVA: Discovering Well-Conditioned Winograd Transforms through Numerical Optimization of Vandermonde Arithmetic](https://mlsanity.com/paper/2512.18453/) (2025)

## Abstract (arXiv)

Convolutional Neural Networks (CNNs) are computationally intensive, which limits their application on mobile devices. Their energy is dominated by the number of multiplies needed to perform the convolutions. Winograd's minimal filtering algorithm (Lavin, 2015) and network pruning (Han et al., 2015) can reduce the operation count, but these two methods cannot be directly combined - applying the Winograd transform fills in the sparsity in both the weights and the activations. We propose two modifications to Winograd-based CNNs to enable these methods to exploit sparsity. First, we move the ReLU operation into the Winograd domain to increase the sparsity of the transformed activations. Second, we prune the weights in the Winograd domain to exploit static weight sparsity. For models on CIFAR-10, CIFAR-100 and ImageNet datasets, our method reduces the number of multiplications by 10.4×, 6.8× and 10.8× respectively with loss of accuracy less than 0.1%, outperforming previous baselines by 2.0×-3.0×. We also show that moving ReLU to the Winograd domain allows more aggressive pruning.