# Dynamic Binary Neural Network by learning channel-wise thresholds

arXiv: https://arxiv.org/abs/2110.05185 · HTML: https://mlsanity.com/paper/2110.05185/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 21 references that are in the map.
- First posted on arXiv 8 Oct 2021.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients](https://mlsanity.com/paper/1606.06160/) (2016)
- [Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithm](https://mlsanity.com/paper/1808.00278/) (2018)
- [Binary Neural Networks: A Survey](https://mlsanity.com/paper/2004.03333/) (2020)
- [ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functions](https://mlsanity.com/paper/2003.03488/) (2020)
- [Forward and Backward Information Retention for Accurate Binary Neural Networks](https://mlsanity.com/paper/1909.10788/) (2019)
- [Towards Accurate Binary Convolutional Neural Network](https://mlsanity.com/paper/1711.11294/) (2017)
- [Training Binary Neural Networks with Real-to-Binary Convolutions](https://mlsanity.com/paper/2003.11535/) (2020)
- [Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1](https://mlsanity.com/paper/1602.02830/) (2016)
- [XNOR-Net++: Improved Binary Neural Networks](https://mlsanity.com/paper/1909.13863/) (2019)
- [Rotated Binary Neural Network](https://mlsanity.com/paper/2009.13055/) (2020)
- [Dynamic ReLU](https://mlsanity.com/paper/2003.10027/) (2020)
- [How Do Adam and Training Strategies Help BNNs Optimization?](https://mlsanity.com/paper/2106.11309/) (2021)
- [Training Binary Neural Networks through Learning with Noisy Supervision](https://mlsanity.com/paper/2010.04871/) (2020)
- [Projection Convolutional Neural Networks for 1-bit CNNs via Discrete Back Propagation](https://mlsanity.com/paper/1811.12755/) (2018)
- [Bayesian Optimized 1-Bit CNNs](https://mlsanity.com/paper/1908.06314/) (2019)

## Built on by (most cited first, 13 of 13)

- [A comprehensive review of Binary Neural Network](https://mlsanity.com/paper/2110.06804/) (2021)
- [Lightweight Pixel Difference Networks for Efficient Visual Representation Learning](https://mlsanity.com/paper/2402.00422/) (2024)
- [BEBERT: Efficient and Robust Binary Ensemble BERT](https://mlsanity.com/paper/2210.15976/) (2022)
- [SVNet: Where SO(3) Equivariance Meets Binarization on Point Cloud Representation](https://mlsanity.com/paper/2209.05924/) (2022)
- [Rapid Salient Object Detection with Difference Convolutional Neural Networks](https://mlsanity.com/paper/2507.01182/) (2025)
- [INSTA-BNN: Binary Neural Network with INSTAnce-aware Threshold](https://mlsanity.com/paper/2204.07439/) (2022)
- [Towards a Smaller Student: Capacity Dynamic Distillation for Efficient Image Retrieval](https://mlsanity.com/paper/2303.09230/) (2023)
- [Boosting Convolutional Neural Networks with Middle Spectrum Grouped Convolution](https://mlsanity.com/paper/2304.06305/) (2023)
- [How to train accurate BNNs for embedded systems?](https://mlsanity.com/paper/2206.12322/) (2022)
- [From Local Binary Patterns to Pixel Difference Networks for Efficient Visual Representation Learning](https://mlsanity.com/paper/2303.08414/) (2023)
- [Boosting Binary Neural Networks via Dynamic Thresholds Learning](https://mlsanity.com/paper/2211.02292/) (2022)
- [Designed Dithering Sign Activation for Binary Neural Networks](https://mlsanity.com/paper/2405.02220/) (2024)
- [BinaryDemoire: Moir'e-Aware Binarization for Image Demoir'eing](https://mlsanity.com/paper/2602.03176/) (2026)

## Abstract (arXiv)

Binary neural networks (BNNs) constrain weights and activations to +1 or -1 with limited storage and computational cost, which is hardware-friendly for portable devices. Recently, BNNs have achieved remarkable progress and been adopted into various fields. However, the performance of BNNs is sensitive to activation distribution. The existing BNNs utilized the Sign function with predefined or learned static thresholds to binarize activations. This process limits representation capacity of BNNs since different samples may adapt to unequal thresholds. To address this problem, we propose a dynamic BNN (DyBNN) incorporating dynamic learnable channel-wise thresholds of Sign function and shift parameters of PReLU. The method aggregates the global information into the hyper function and effectively increases the feature expression ability. The experimental results prove that our method is an effective and straightforward way to reduce information loss and enhance performance of BNNs. The DyBNN based on two backbones of ReActNet (MobileNetV1 and ResNet18) achieve 71.2% and 67.4% top1-accuracy on ImageNet dataset, outperforming baselines by a large margin (i.e., 1.8% and 1.5% respectively).