# AdaBin: Improving Binary Neural Networks with Adaptive Binary Sets

arXiv: https://arxiv.org/abs/2208.08084 · HTML: https://mlsanity.com/paper/2208.08084/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 54 references that are in the map.
- First posted on arXiv 17 Aug 2022.

## Builds on (most cited first, 30 of 41)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients](https://mlsanity.com/paper/1606.06160/) (2016)
- [GhostNet: More Features from Cheap Operations](https://mlsanity.com/paper/1911.11907/) (2019)
- [Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networks](https://mlsanity.com/paper/1908.05033/) (2019)
- [Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithm](https://mlsanity.com/paper/1808.00278/) (2018)
- [Ternary Weight Networks](https://mlsanity.com/paper/1605.04711/) (2016)
- [Low-bit Quantization of Neural Networks for Efficient Inference](https://mlsanity.com/paper/1902.06822/) (2019)
- [Maxout Networks](https://mlsanity.com/paper/1302.4389/) (2013)
- [ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functions](https://mlsanity.com/paper/2003.03488/) (2020)
- [Forward and Backward Information Retention for Accurate Binary Neural Networks](https://mlsanity.com/paper/1909.10788/) (2019)
- [Towards Accurate Binary Convolutional Neural Network](https://mlsanity.com/paper/1711.11294/) (2017)
- [Deep Learning with Low Precision by Half-wave Gaussian Quantization](https://mlsanity.com/paper/1702.00953/) (2017)
- [Training Binary Neural Networks with Real-to-Binary Convolutions](https://mlsanity.com/paper/2003.11535/) (2020)
- [Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1](https://mlsanity.com/paper/1602.02830/) (2016)
- [AdderNet: Do We Really Need Multiplications in Deep Learning?](https://mlsanity.com/paper/1912.13200/) (2019)
- [Rotated Binary Neural Network](https://mlsanity.com/paper/2009.13055/) (2020)
- [Regularizing Activation Distribution for Training Binarized Deep Networks](https://mlsanity.com/paper/1904.02823/) (2019)
- [Loss-aware Binarization of Deep Networks](https://mlsanity.com/paper/1611.01600/) (2016)
- [ReCU: Reviving the Dead Weights in Binary Neural Networks](https://mlsanity.com/paper/2103.12369/) (2021)
- [Searching for Low-Bit Weights in Quantized Neural Networks](https://mlsanity.com/paper/2009.08695/) (2020)
- [Training Binary Neural Networks through Learning with Noisy Supervision](https://mlsanity.com/paper/2010.04871/) (2020)
- [Learning Frequency Domain Approximation for Binary Neural Networks](https://mlsanity.com/paper/2103.00841/) (2021)

## Built on by (most cited first, 30 of 38)

- [Convolutional Differentiable Logic Gate Networks](https://mlsanity.com/paper/2411.04732/) (2024)
- [BiViT: Extremely Compressed Binary Vision Transformer](https://mlsanity.com/paper/2211.07091/) (2022)
- [Faster Than Lies: Real-time Deepfake Detection using Binary Neural Networks](https://mlsanity.com/paper/2406.04932/) (2024)
- [QuEST: Stable Training of LLMs with 1-Bit Weights and Activations](https://mlsanity.com/paper/2502.05003/) (2025)
- [Estimator Meets Equilibrium Perspective: A Rectified Straight Through Estimator for Binary Neural Networks Training](https://mlsanity.com/paper/2308.06689/) (2023)
- [Join the High Accuracy Club on ImageNet with A Binary Neural Network Ticket](https://mlsanity.com/paper/2211.12933/) (2022)
- [Improving the Straight-Through Estimator with Zeroth-Order Information](https://mlsanity.com/paper/2510.23926/) (2025)
- [SCALES: Boost Binary Neural Network for Image Super-Resolution with Efficient Scalings](https://mlsanity.com/paper/2303.12270/) (2023)
- [Understanding Neural Network Binarization with Forward and Backward Proximal Quantizers](https://mlsanity.com/paper/2402.17710/) (2024)
- [Outlier-Aware Post-Training Quantization for Image Super-Resolution](https://mlsanity.com/paper/2511.00682/) (2025)
- [BiMLP: Compact Binary Architectures for Vision Multi-Layer Perceptrons](https://mlsanity.com/paper/2212.14158/) (2022)
- [Compacting Binary Neural Networks by Sparse Kernel Selection](https://mlsanity.com/paper/2303.14470/) (2023)
- [BOLD: Boolean Logic Deep Learning](https://mlsanity.com/paper/2405.16339/) (2024)
- [Binary Weight Multi-Bit Activation Quantization for Compute-in-Memory CNN Accelerators](https://mlsanity.com/paper/2508.21524/) (2025)
- [AskewSGD : An Annealed interval-constrained Optimisation method to train Quantized Neural Networks](https://mlsanity.com/paper/2211.03741/) (2022)
- [OvSW: Overcoming Silent Weights for Accurate Binary Neural Networks](https://mlsanity.com/paper/2407.05257/) (2024)
- [Information-Bottleneck Driven Binary Neural Network for Change Detection](https://mlsanity.com/paper/2507.03504/) (2025)
- [BiDense: Binarization for Dense Prediction](https://mlsanity.com/paper/2411.10346/) (2024)
- [Training Multi-Layer Binary Neural Networks With Local Binary Error Signals](https://mlsanity.com/paper/2412.00119/) (2024)
- [10K is Enough: An Ultra-Lightweight Binarized Network for Infrared Small-Target Detection](https://mlsanity.com/paper/2503.02662/) (2025)
- [Rethinking 1-bit Optimization Leveraging Pre-trained Large Language Models](https://mlsanity.com/paper/2508.06974/) (2025)
- [MST-compression: Compressing and Accelerating Binary Neural Networks with Minimum Spanning Tree](https://mlsanity.com/paper/2308.13735/) (2023)
- [Hamming Encoder: Mining Discriminative k-mers for Discrete Sequence Classification](https://mlsanity.com/paper/2310.10321/) (2023)
- [Quantized Embedding Vectors for Controllable Diffusion Language Models](https://mlsanity.com/paper/2402.10107/) (2024)
- [BDC-Occ: Binarized Deep Convolution Unit For Binarized Occupancy Network](https://mlsanity.com/paper/2405.17037/) (2024)
- [CycleBNN: Cyclic Precision Training in Binary Neural Networks](https://mlsanity.com/paper/2410.00050/) (2024)
- [Efficiera Residual Networks: Hardware-Friendly Fully Binary Weight with 2-bit Activation Model Achieves Practical ImageNet Accuracy](https://mlsanity.com/paper/2410.11553/) (2024)
- [Leveraging Stable Diffusion for Monocular Depth Estimation via Image Semantic Encoding](https://mlsanity.com/paper/2502.01666/) (2025)
- [Expanding-and-Shrinking Binary Neural Networks](https://mlsanity.com/paper/2503.23709/) (2025)
- [High-Fidelity Differential-information Driven Binary Vision Transformer](https://mlsanity.com/paper/2507.02222/) (2025)

## Abstract (arXiv)

This paper studies the Binary Neural Networks (BNNs) in which weights and activations are both binarized into 1-bit values, thus greatly reducing the memory usage and computational complexity. Since the modern deep neural networks are of sophisticated design with complex architecture for the accuracy reason, the diversity on distributions of weights and activations is very high. Therefore, the conventional sign function cannot be well used for effectively binarizing full-precision values in BNNs. To this end, we present a simple yet effective approach called AdaBin to adaptively obtain the optimal binary sets b1, b2 (b1, b2∈ ℝ) of weights and activations for each layer instead of a fixed set (textiti.e., -1, +1). In this way, the proposed method can better fit different distributions and increase the representation ability of binarized features. In practice, we use the center position and distance of 1-bit values to define a new binary quantization function. For the weights, we propose an equalization method to align the symmetrical center of binary distribution to real-valued distribution, and minimize the Kullback-Leibler divergence of them. Meanwhile, we introduce a gradient-based optimization method to get these two parameters for activations, which are jointly trained in an end-to-end manner. Experimental results on benchmark models and datasets demonstrate that the proposed AdaBin is able to achieve state-of-the-art performance. For instance, we obtain a 66.4% Top-1 accuracy on the ImageNet using ResNet-18 architecture, and a 69.4 mAP on PASCAL VOC using SSD300. The PyTorch code is available at urlhttps://github.com/huawei-noah/Efficient-Computing/tree/master/BinaryNetworks/AdaBin and the MindSpore code is available at urlhttps://gitee.com/mindspore/models/tree/master/research/cv/AdaBin.