# Bayesian Optimized 1-Bit CNNs

arXiv: https://arxiv.org/abs/1908.06314 · HTML: https://mlsanity.com/paper/1908.06314/

## Key facts

- Cited by 14 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 17 Aug 2019.

## Built on by (most cited first, 14 of 14)

- [ReCU: Reviving the Dead Weights in Binary Neural Networks](https://mlsanity.com/paper/2103.12369/) (2021)
- [A comprehensive review of Binary Neural Network](https://mlsanity.com/paper/2110.06804/) (2021)
- [Learning Frequency Domain Approximation for Binary Neural Networks](https://mlsanity.com/paper/2103.00841/) (2021)
- [Distribution-sensitive Information Retention for Accurate Binary Neural Network](https://mlsanity.com/paper/2109.12338/) (2021)
- [Estimator Meets Equilibrium Perspective: A Rectified Straight Through Estimator for Binary Neural Networks Training](https://mlsanity.com/paper/2308.06689/) (2023)
- [SiMaN: Sign-to-Magnitude Network Binarization](https://mlsanity.com/paper/2102.07981/) (2021)
- [Dynamic Binary Neural Network by learning channel-wise thresholds](https://mlsanity.com/paper/2110.05185/) (2021)
- [BiPer: Binary Neural Networks using a Periodic Function](https://mlsanity.com/paper/2404.01278/) (2024)
- [POEM: 1-bit Point-wise Operations based on Expectation-Maximization for Efficient Point Cloud Processing](https://mlsanity.com/paper/2111.13386/) (2021)
- [A&B BNN: Add&Bit-Operation-Only Hardware-Friendly Binary Neural Network](https://mlsanity.com/paper/2403.03739/) (2024)
- [Improving Binary Neural Networks through Fully Utilizing Latent Weights](https://mlsanity.com/paper/2110.05850/) (2021)
- [BNAS v2: Learning Architectures for Binary Networks with Empirical Improvements](https://mlsanity.com/paper/2110.08562/) (2021)
- [E2FIF: Push the limit of Binarized Deep Imagery Super-resolution using End-to-end Full-precision Information Flow](https://mlsanity.com/paper/2207.06893/) (2022)
- [Boosting Binary Neural Networks via Dynamic Thresholds Learning](https://mlsanity.com/paper/2211.02292/) (2022)

## Abstract (arXiv)

Deep convolutional neural networks (DCNNs) have dominated the recent developments in computer vision through making various record-breaking models. However, it is still a great challenge to achieve powerful DCNNs in resource-limited environments, such as on embedded devices and smart phones. Researchers have realized that 1-bit CNNs can be one feasible solution to resolve the issue; however, they are baffled by the inferior performance compared to the full-precision DCNNs. In this paper, we propose a novel approach, called Bayesian optimized 1-bit CNNs (denoted as BONNs), taking the advantage of Bayesian learning, a well-established strategy for hard problems, to significantly improve the performance of extreme 1-bit CNNs. We incorporate the prior distributions of full-precision kernels and features into the Bayesian framework to construct 1-bit CNNs in an end-to-end manner, which have not been considered in any previous related methods. The Bayesian losses are achieved with a theoretical support to optimize the network simultaneously in both continuous and discrete spaces, aggregating different losses jointly to improve the model capacity. Extensive experiments on the ImageNet and CIFAR datasets show that BONNs achieve the best classification performance compared to state-of-the-art 1-bit CNNs.