mlsanityData to 14 Oct 2026

Sparsifying Binary Networks

arXiv 2207.04974 ↗First posted 11 Jul 2022cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Binary neural networks (BNNs) have demonstrated their ability to solve complex tasks with comparable accuracy as full-precision deep neural networks (DNNs), while also reducing computational power and storage requirements and increasing the processing speed. These properties make them an attractive alternative for the development and deployment of DNN-based applications in Internet-of-Things (IoT) devices. Despite the recent improvements, they suffer from a fixed and limited compression factor that may result insufficient for certain devices with very limited resources. In this work, we propose sparse binary neural networks (SBNNs), a novel model and training scheme which introduces sparsity in BNNs and a new quantization function for binarizing the network's weights. The proposed SBNN is able to achieve high compression factors and it reduces the number of operations and parameters at inference time. We also provide tools to assist the SBNN design, while respecting hardware resource constraints. We study the generalization properties of our method for different compression factors through a set of experiments on linear and convolutional networks on three datasets. Our experiments confirm that SBNNs can achieve high compression rates, without compromising generalization, while further reducing the operations of BNNs, making SBNNs a viable option for deploying DNNs in cheap, low-cost, limited-resources IoT devices and sensors.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 35 of its 54 references that are in the map. First posted on arXiv 11 Jul 2022.

Builds on

35 of 54 references.

References · 35

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  5. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  7. 2014Going Deeper with Convolutionscited by 4,241
  8. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  9. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  10. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  11. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  12. 2016Identity Mappings in Deep Residual Networkscited by 1,965
23 more
  1. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,136
  2. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 849
  3. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 592
  4. 2017Learning Sparse Neural Networks through L0 Regularizationcited by 583
  5. 2013Do Deep Nets Really Need to be Deep?cited by 474
  6. 2017Efficient Processing of Deep Neural Networks: A Tutorial and Surveycited by 434
  7. 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 426
  8. 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 274
  9. 2020MCUNet: Tiny Deep Learning on IoT Devicescited by 273
  10. 2018Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithmcited by 235
  11. 2013Predicting Parameters in Deep Learningcited by 224
  12. 2020ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functionscited by 184
  13. 2019Quantization Networkscited by 153
  14. 2016FINN: A Framework for Fast, Scalable Binarized Neural Network Inferencecited by 150
  15. 2016An Analysis of Deep Neural Network Models for Practical Applicationscited by 116
  16. 2016Training Sparse Neural Networkscited by 73
  17. 2019Regularizing Activation Distribution for Training Binarized Deep Networkscited by 61
  18. 2019Learning Sparse Networks Using Targeted Dropoutcited by 40
  19. 2016Bitwise Neural Networkscited by 29
  20. 2019Back to Simplicity: How to Train Accurate BNNs from Scratch?cited by 24
  21. 2020Learning Sparse & Ternary Neural Networks with Entropy-Constrained Trained Ternarization (EC2T)cited by 6
  22. 2017A Review on Internet of Things (IoT), Internet of Everything (IoE) and Internet of Nano Things (IoNT)cited by 4
  23. 2017Compressing Low Precision Deep Neural Networks Using Sparsity-Induced Regularization in Ternary Networkscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Binary domain generalization for sparsifying binary neural networks
  2. 2021Sub-bit Neural Networks: Learning to Compress and Accelerate Binary Neural Networks
  3. 2022Exploiting Kernel Compression on BNNs
  4. 2024Tiled Bit Networks: Sub-Bit Neural Network Compression Through Reuse of Learnable Binary Vectors
  5. 2023Compacting Binary Neural Networks by Sparse Kernel Selection
  6. 2021A comprehensive review of Binary Neural Network
  7. 2021Enabling Binary Neural Network Training on the Edge
  8. 2025BD-Net: Has Depth-Wise Convolution Ever Been Applied in Binary Neural Networks?