mlsanityData to 14 Oct 2026

Winograd Algorithm for AdderNet

arXiv 2105.05530 ↗First posted 12 May 2021cs.LG cs.CV cs.NEcited by 4

Not on the map. Why?

Abstract (arXiv)

Adder neural network (AdderNet) is a new kind of deep model that replaces the original massive multiplications in convolutions by additions while preserving the high performance. Since the hardware complexity of additions is much lower than that of multiplications, the overall energy consumption is thus reduced significantly. To further optimize the hardware overhead of using AdderNet, this paper studies the winograd algorithm, which is a widely used fast algorithm for accelerating convolution and saving the computational costs. Unfortunately, the conventional Winograd algorithm cannot be directly applied to AdderNets since the distributive law in multiplication is not valid for the l1-norm. Therefore, we replace the element-wise multiplication in the Winograd equation by additions and then develop a new set of transform matrixes that can enhance the representation ability of output features to maintain the performance. Moreover, we propose the l2-to-l1 training strategy to mitigate the negative impacts caused by formal inconsistency. Experimental results on both FPGA and benchmarks show that the new method can further reduce the energy consumption without affecting the accuracy of the original AdderNet.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 19 of its 27 references that are in the map. First posted on arXiv 12 May 2021.

Builds on

19 of 27 references.

References · 19

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  4. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  5. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  6. 2019EfficientDet: Scalable and Efficient Object Detectioncited by 1,020
  7. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 849
  8. 2019GhostNet: More Features from Cheap Operationscited by 410
  9. 2018Toward Convolutional Blind Denoising of Real Photographscited by 274
  10. 2019Progressive Image Deraining Networks: A Better and Simpler Baselinecited by 272
  11. 2020ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functionscited by 184
  12. 2020Hierarchical Multi-Scale Attention for Semantic Segmentationcited by 183
7 more
  1. 2017Towards Accurate Binary Convolutional Neural Networkcited by 149
  2. 2018Densely Connected Pyramid Dehazing Networkcited by 145
  3. 2015Fast Algorithms for Convolutional Neural Networkscited by 102
  4. 2013Fast Training of Convolutional Networks through FFTscited by 89
  5. 2019AdderNet: Do We Really Need Multiplications in Deep Learning?cited by 82
  6. 2020AdderSR: Towards Energy Efficient Image Super-Resolutioncited by 32
  7. 2020Kernel Based Progressive Distillation for Adder Neural Networkscited by 19

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 3.

4 papers.

  1. 2022Hoyer regularizer is all you need for ultra low-latency spiking neural networkscited by 4
  2. 2022Redistribution of Weights and Activations for AdderNet Quantizationcited by 3
  3. 2022Going Further With Winograd Convolutions: Tap-Wise Quantization for Efficient Inference on 4x4 Tilecited by 1
  4. 2021Fast Convolution based on Winograd Minimum Filtering: Introduction and Developmentcited by 0