mlsanityData to 14 Oct 2026

LAB: Learnable Activation Binarizer for Binary Neural Networks

arXiv 2210.13858 ↗First posted 25 Oct 2022cs.LG cs.AI cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Binary Neural Networks (BNNs) are receiving an upsurge of attention for bringing power-hungry deep learning towards edge devices. The traditional wisdom in this space is to employ sign() for binarizing featuremaps. We argue and illustrate that sign() is a uniqueness bottleneck, limiting information propagation throughout the network. To alleviate this, we propose to dispense sign(), replacing it with a learnable activation binarizer (LAB), allowing the network to learn a fine-grained binarization kernel per layer - as opposed to global thresholding. LAB is a novel universal module that can seamlessly be integrated into existing architectures. To confirm this, we plug it into four seminal BNNs and show a considerable performance boost at the cost of tolerable increase in delay and complexity. Finally, we build an end-to-end BNN (coined as LAB-BNN) around LAB, and demonstrate that it achieves competitive performance on par with the state-of-the-art on ImageNet.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 29 of its 41 references that are in the map. First posted on arXiv 25 Oct 2022.

Builds on

29 of 41 references.

References · 29

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  3. 2020Knowledge Distillation: A Surveycited by 1,603
  4. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 849
  5. 2018Rethinking the Value of Network Pruningcited by 599
  6. 2018Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithmcited by 235
  7. 2020ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functionscited by 184
  8. 2019Forward and Backward Information Retention for Accurate Binary Neural Networkscited by 169
  9. 2015Deep Spatial Autoencoders for Visuomotor Learningcited by 153
  10. 2020Training Binary Neural Networks with Real-to-Binary Convolutionscited by 112
  11. 2020Rotated Binary Neural Networkcited by 77
  12. 2019Regularizing Activation Distribution for Training Binarized Deep Networkscited by 61
17 more
  1. 2016Loss-aware Binarization of Deep Networkscited by 56
  2. 2021PokeBNN: A Binary Pursuit of Lightweight Accuracycited by 49
  3. 2021How Do Adam and Training Strategies Help BNNs Optimization?cited by 46
  4. 2020Training Binary Neural Networks through Learning with Noisy Supervisioncited by 39
  5. 2020High-Capacity Expert Binary Networkscited by 34
  6. 2018Deep Neural Network Compression with Single and Multiple Level Quantizationcited by 27
  7. 2019daBNN: A Super Fast Inference Framework for Binary Neural Networks on ARM devicescited by 27
  8. 2020BATS: Binary ArchitecTure Searchcited by 26
  9. 2021Distribution-sensitive Information Retention for Accurate Binary Neural Networkcited by 25
  10. 2019Back to Simplicity: How to Train Accurate BNNs from Scratch?cited by 24
  11. 2019Improved training of binary networks for human pose estimation and image recognitioncited by 22
  12. 2020Larq Compute Engine: Design, Benchmark, and Deploy State-of-the-Art Binarized Neural Networkscited by 20
  13. 2021SiMaN: Sign-to-Magnitude Network Binarizationcited by 18
  14. 2021"BNN - BN = ?": Training Binary Neural Networks without Batch Normalizationcited by 17
  15. 2020Improving Accuracy of Binary Neural Networks using Unbalanced Activation Distributioncited by 15
  16. 2017BMXNet: An Open-Source Binary Neural Network Implementation Based on MXNetcited by 11
  17. 2020NASB: Neural Architecture Search for Binary Convolutional Neural Networkscited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022Boosting Binary Neural Networks via Dynamic Thresholds Learning
  2. 2024Designed Dithering Sign Activation for Binary Neural Networks
  3. 2022Binarizing by Classification: Is soft function really necessary?
  4. 2024Understanding Neural Network Binarization with Forward and Backward Proximal Quantizers
  5. 2023Input Layer Binarization with Bit-Plane Encoding
  6. 2021Enabling Binary Neural Network Training on the Edge
  7. 2022Join the High Accuracy Club on ImageNet with A Binary Neural Network Ticket
  8. 2022AdaBin: Improving Binary Neural Networks with Adaptive Binary Sets