mlsanityData to 7 Oct 2026

Self-Distribution Binary Neural Networks

arXiv 2103.02394 ↗First posted 3 Mar 2021cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

In this work, we study the binary neural networks (BNNs) of which both the weights and activations are binary (i.e., 1-bit representation). Feature representation is critical for deep neural networks, while in BNNs, the features only differ in signs. Prior work introduces scaling factors into binary weights and activations to reduce the quantization error and effectively improves the classification accuracy of BNNs. However, the scaling factors not only increase the computational complexity of networks, but also make no sense to the signs of binary features. To this end, Self-Distribution Binary Neural Network (SD-BNN) is proposed. Firstly, we utilize Activation Self Distribution (ASD) to adaptively adjust the sign distribution of activations, thereby improve the sign differences of the outputs of the convolution. Secondly, we adjust the sign distribution of weights through Weight Self Distribution (WSD) and then fine-tune the sign distribution of the outputs of the convolution. Extensive experiments on CIFAR-10 and ImageNet datasets with various network structures show that the proposed SD-BNN consistently outperforms the state-of-the-art (SOTA) BNNs (e.g., achieves 92.5% on CIFAR-10 and 66.5% on ImageNet with ResNet-18) with less computation cost. Code is available at https://github.com/ pingxue-hfut/SD-BNN.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 36 of its 58 references that are in the map. First posted on arXiv 3 Mar 2021.

Builds on

36 of 58 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2017Squeeze-and-Excitation Networkscited by 4,763
  3. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  4. 2015Fast R-CNNcited by 2,930
  5. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  6. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  7. 2020Knowledge Distillation: A Surveycited by 1,578
  8. 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,059
  9. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
  10. 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 413
  11. 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 272
  12. 2019Libra R-CNN: Towards Balanced Learning for Object Detectioncited by 255
24 more
  1. 2019Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networkscited by 234
  2. 2018Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithmcited by 233
  3. 2019Forward and Backward Information Retention for Accurate Binary Neural Networkscited by 169
  4. 2017Towards Accurate Binary Convolutional Neural Networkcited by 148
  5. 2020Training Binary Neural Networks with Real-to-Binary Convolutionscited by 111
  6. 2019XNOR-Net++: Improved Binary Neural Networkscited by 94
  7. 2019Dynamic Curriculum Learning for Imbalanced Data Classificationcited by 93
  8. 2020Rotated Binary Neural Networkcited by 77
  9. 2019Regularizing Activation Distribution for Training Binarized Deep Networkscited by 61
  10. 2018Bi-Real Net: Binarizing Deep Network Towards Real-Network Performancecited by 42
  11. 2018Binary Ensemble Neural Network: More Bits per Network or More Networks per Bit?cited by 38
  12. 2018Structured Binary Neural Networks for Accurate Image Classification and Semantic Segmentationcited by 37
  13. 2019Circulant Binary Convolutional Networks: Enhancing the Performance of 1-bit DCNNs with Circulant Back Propagationcited by 28
  14. 2020ResKD: Residual-Guided Knowledge Distillationcited by 25
  15. 2017Performance Guaranteed Network Acceleration via High-Order Residual Quantizationcited by 24
  16. 2020BinaryDuo: Reducing Gradient Mismatch in Binary Activation Network by Coupling Binary Activationscited by 23
  17. 2019Improved training of binary networks for human pose estimation and image recognitioncited by 22
  18. 2020Least squares binary quantization of neural networkscited by 21
  19. 2020Weakly Supervised Attention Pyramid Convolutional Neural Network for Fine-Grained Visual Classificationcited by 18
  20. 2020Depth Quality Aware Salient Object Detectioncited by 17
  21. 2019Balanced Binary Neural Networks with Gated Residualcited by 14
  22. 2018ChannelNets: Compact and Efficient Convolutional Neural Networks via Channel-Wise Convolutionscited by 10
  23. 2019On Symbiosis of Attribute Prediction and Semantic Segmentationcited by 6
  24. 2020Compressing Deep Convolutional Neural Networks by Stacking Low-dimensional Binary Convolution Filterscited by 5

Built on by

Cited by year

Citing papers per year: 2022: 4.

4 papers.

  1. 2022PECAN: A Product-Quantized Content Addressable Memory Networkcited by 6
  2. 2022Bimodal Distributed Binarized Neural Networkscited by 4
  3. 2022Binarizing by Classification: Is soft function really necessary?cited by 2
  4. 2022How to train accurate BNNs for embedded systems?cited by 1