mlsanityData to 7 Oct 2026

Signed Binary Weight Networks

arXiv 2211.13838 ↗First posted 25 Nov 2022cs.CV cs.DC cs.PFcited by 1

Not on the map. Why?

Abstract (arXiv)

Efficient inference of Deep Neural Networks (DNNs) is essential to making AI ubiquitous. Two important algorithmic techniques have shown promise for enabling efficient inference - sparsity and binarization. These techniques translate into weight sparsity and weight repetition at the hardware-software level enabling the deployment of DNNs with critically low power and latency requirements. We propose a new method called signed-binary networks to improve efficiency further (by exploiting both weight sparsity and weight repetition together) while maintaining similar accuracy. Our method achieves comparable accuracy on ImageNet and CIFAR10 datasets with binary and can lead to 69% sparsity. We observe real speedup when deploying these models on general-purpose devices and show that this high percentage of unstructured sparsity can lead to a further reduction in energy consumption on ASICs.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 34 of its 50 references that are in the map. First posted on arXiv 25 Nov 2022.

Builds on

34 of 50 references.

References · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  7. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  9. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  10. 2022A ConvNet for the 2020scited by 3,545
  11. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  12. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
22 more
  1. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
  2. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 666
  3. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 571
  4. 2016EIE: Efficient Inference Engine on Compressed Deep Neural Networkcited by 291
  5. 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 272
  6. 2019Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networkscited by 234
  7. 2018Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithmcited by 233
  8. 2020SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruningcited by 228
  9. 2016Ternary Weight Networkscited by 195
  10. 2020ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functionscited by 183
  11. 2019Forward and Backward Information Retention for Accurate Binary Neural Networkscited by 169
  12. 2017Towards Accurate Binary Convolutional Neural Networkcited by 148
  13. 2019XNOR-Net++: Improved Binary Neural Networkscited by 94
  14. 2021Sparse is Enough in Scaling Transformerscited by 86
  15. 2018ProxQuant: Quantized Neural Networks via Proximal Operatorscited by 56
  16. 2021ReCU: Reviving the Dead Weights in Binary Neural Networkscited by 51
  17. 2018From Hashing to CNNs: Training BinaryWeight Networks via Hashingcited by 23
  18. 2019Improved training of binary networks for human pose estimation and image recognitioncited by 22
  19. 2020Least squares binary quantization of neural networkscited by 21
  20. 2018UCNN: Exploiting Computational Reuse in Deep Neural Networks via Weight Repetitioncited by 19
  21. 2020SparseTrain: Exploiting Dataflow Sparsity for Efficient Convolutional Neural Networks Trainingcited by 10
  22. 2019Binarized Neural Architecture Searchcited by 6

Built on by

Cited by year

Citing papers per year: 2023: 1.

1 paper.

  1. 2023Binary domain generalization for sparsifying binary neural networkscited by 1