# Searching for Winograd-aware Quantized Networks

arXiv: https://arxiv.org/abs/2002.10711 · HTML: https://mlsanity.com/paper/2002.10711/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Feb 2020.

## Built on by (most cited first, 11 of 11)

- [A Comprehensive Survey on Hardware-Aware Neural Architecture Search](https://mlsanity.com/paper/2101.09336/) (2021)
- [Smart at what cost? Characterising Mobile Deep Neural Networks in the wild](https://mlsanity.com/paper/2109.13963/) (2021)
- [Winograd Convolution for Deep Neural Networks: Efficient Point Selection](https://mlsanity.com/paper/2201.10369/) (2022)
- [Winograd Convolution: A Perspective from Fault Tolerance](https://mlsanity.com/paper/2202.08675/) (2022)
- [unzipFPGA: Enhancing FPGA-based CNN Engines with On-the-Fly Weights Generation](https://mlsanity.com/paper/2103.05600/) (2021)
- [Going Further With Winograd Convolutions: Tap-Wise Quantization for Efficient Inference on 4x4 Tile](https://mlsanity.com/paper/2209.12982/) (2022)
- [Mitigating Memory Wall Effects in CNN Engines with On-the-Fly Weights Generation](https://mlsanity.com/paper/2307.13412/) (2023)
- [YFlows: Systematic Dataflow Exploration and Code Generation for Efficient Neural Network Inference using SIMD Architectures on CPUs](https://mlsanity.com/paper/2310.00574/) (2023)
- [Data-Free Group-Wise Fully Quantized Winograd Convolution via Learnable Scales](https://mlsanity.com/paper/2412.19867/) (2024)
- [Fast Convolution based on Winograd Minimum Filtering: Introduction and Development](https://mlsanity.com/paper/2111.00977/) (2021)
- [A Survey on Multi-Objective Neural Architecture Search](https://mlsanity.com/paper/2307.09099/) (2023)

## Abstract (arXiv)

Lightweight architectural designs of Convolutional Neural Networks (CNNs) together with quantization have paved the way for the deployment of demanding computer vision applications on mobile devices. Parallel to this, alternative formulations to the convolution operation such as FFT, Strassen and Winograd, have been adapted for use in CNNs offering further speedups. Winograd convolutions are the fastest known algorithm for spatially small convolutions, but exploiting their full potential comes with the burden of numerical error, rendering them unusable in quantized contexts. In this work we propose a Winograd-aware formulation of convolution layers which exposes the numerical inaccuracies introduced by the Winograd transformations to the learning of the model parameters, enabling the design of competitive quantized models without impacting model size. We also address the source of the numerical error and propose a relaxation on the form of the transformation matrices, resulting in up to 10% higher classification accuracy on CIFAR-10. Finally, we propose wiNAS, a neural architecture search (NAS) framework that jointly optimizes a given macro-architecture for accuracy and latency leveraging Winograd-aware layers. A Winograd-aware ResNet-18 optimized with wiNAS for CIFAR-10 results in 2.66x speedup compared to im2row, one of the most widely used optimized convolution implementations, with no loss in accuracy.