# Winograd Convolution for Deep Neural Networks: Efficient Point Selection

arXiv: https://arxiv.org/abs/2201.10369 · HTML: https://mlsanity.com/paper/2201.10369/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 37 references that are in the map.
- First posted on arXiv 25 Jan 2022.

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [Caffe: Convolutional Architecture for Fast Feature Embedding](https://mlsanity.com/paper/1408.5093/) (2014)
- [Fast Algorithms for Convolutional Neural Networks](https://mlsanity.com/paper/1509.09308/) (2015)
- [Searching for Winograd-aware Quantized Networks](https://mlsanity.com/paper/2002.10711/) (2020)
- [FFT-Based Deep Learning Deployment in Embedded Systems](https://mlsanity.com/paper/1712.04910/) (2017)
- [Error Analysis and Improving the Accuracy of Winograd Convolution for Deep Neural Networks](https://mlsanity.com/paper/1803.10986/) (2018)
- [Efficient Winograd or Cook-Toom Convolution Kernel Implementation on Widely Used Mobile CPUs](https://mlsanity.com/paper/1903.01521/) (2019)
- [DWM: A Decomposable Winograd Method for Convolution Acceleration](https://mlsanity.com/paper/2002.00552/) (2020)
- [Winograd Convolution for DNNs: Beyond linear polynomials](https://mlsanity.com/paper/1905.05233/) (2019)

## Built on by (most cited first, 8 of 8)

- [Speed Is All You Need: On-Device Acceleration of Large Diffusion Models via GPU-Aware Optimizations](https://mlsanity.com/paper/2304.11267/) (2023)
- [Efficient ML Models for Practical Secure Inference](https://mlsanity.com/paper/2209.00411/) (2022)
- [Going Further With Winograd Convolutions: Tap-Wise Quantization for Efficient Inference on 4x4 Tile](https://mlsanity.com/paper/2209.12982/) (2022)
- [Low-Rank Winograd Transformation for 3D Convolutional Neural Networks](https://mlsanity.com/paper/2301.11180/) (2023)
- [YFlows: Systematic Dataflow Exploration and Code Generation for Efficient Neural Network Inference using SIMD Architectures on CPUs](https://mlsanity.com/paper/2310.00574/) (2023)
- [SFC: Achieve Accurate Fast Convolution under Low-precision Arithmetic](https://mlsanity.com/paper/2407.02913/) (2024)
- [Data-Free Group-Wise Fully Quantized Winograd Convolution via Learnable Scales](https://mlsanity.com/paper/2412.19867/) (2024)
- [NOVA: Discovering Well-Conditioned Winograd Transforms through Numerical Optimization of Vandermonde Arithmetic](https://mlsanity.com/paper/2512.18453/) (2025)

## Abstract (arXiv)

Convolutional neural networks (CNNs) have dramatically improved the accuracy of tasks such as object recognition, image segmentation and interactive speech systems. CNNs require large amounts of computing resources because ofcomputationally intensive convolution layers. Fast convolution algorithms such as Winograd convolution can greatly reduce the computational cost of these layers at a cost of poor numeric properties, such that greater savings in computation exponentially increase floating point errors. A defining feature of each Winograd convolution algorithm is a set of real-value points where polynomials are sampled. The choice of points impacts the numeric accuracy of the algorithm, but the optimal set of points for small convolutions remains unknown. Existing work considers only small integers and simple fractions as candidate points. In this work, we propose a novel approach to point selection using points of the form {-1/c , -c, c, 1/c } using the full range of real-valued numbers for c. We show that groups of this form cause cancellations in the Winograd transform matrices that reduce numeric error. We find empirically that the error for different values of c forms a rough curve across the range of real-value numbers helping to localize the values of c that reduce error and that lower errors can be achieved with non-obvious real-valued evaluation points instead of integers or simple fractions. We study a range of sizes for small convolutions and achieve reduction in error ranging from 2% to around 59% for both 1D and 2D convolution. Furthermore, we identify patterns in cases when we select a subset of our proposed points which will always lead to a lower error. Finally we implement a complete Winograd convolution layer and use it to run deep convolution neural networks on real datasets and show that our proposed points reduce error, ranging from 22% to 63%.