# A comprehensive review of Binary Neural Network

arXiv: https://arxiv.org/abs/2110.06804 · HTML: https://mlsanity.com/paper/2110.06804/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 110 of its 190 references that are in the map.
- First posted on arXiv 11 Oct 2021.

## Builds on (most cited first, 30 of 110)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [COCO-Stuff: Thing and Stuff Classes in Context](https://mlsanity.com/paper/1612.03716/) (2016)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [PointCNN: Convolution On X-Transformed Points](https://mlsanity.com/paper/1801.07791/) (2018)
- [PointConv: Deep Convolutional Networks on 3D Point Clouds](https://mlsanity.com/paper/1811.07246/) (2018)
- [DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients](https://mlsanity.com/paper/1606.06160/) (2016)
- [Object Detection in 20 Years: A Survey](https://mlsanity.com/paper/1905.05055/) (2019)
- [Selective Kernel Networks](https://mlsanity.com/paper/1903.06586/) (2019)
- [Deep Learning for Generic Object Detection: A Survey](https://mlsanity.com/paper/1809.02165/) (2018)
- [High-Performance Large-Scale Image Recognition Without Normalization](https://mlsanity.com/paper/2102.06171/) (2021)
- [LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks](https://mlsanity.com/paper/1807.10029/) (2018)

## Built on by (most cited first, 30 of 45)

- [PB-LLM: Partially Binarized Large Language Models](https://mlsanity.com/paper/2310.00034/) (2023)
- [PokeBNN: A Binary Pursuit of Lightweight Accuracy](https://mlsanity.com/paper/2112.00133/) (2021)
- [AutoML in the Age of Large Language Models: Current Challenges, Future Opportunities and Risks](https://mlsanity.com/paper/2306.08107/) (2023)
- [BiBench: Benchmarking and Analyzing Network Binarization](https://mlsanity.com/paper/2301.11233/) (2023)
- [A Brain-Inspired Low-Dimensional Computing Classifier for Inference on Tiny Devices](https://mlsanity.com/paper/2203.04894/) (2022)
- [0/1 Deep Neural Networks via Block Coordinate Descent](https://mlsanity.com/paper/2206.09379/) (2022)
- [BiPer: Binary Neural Networks using a Periodic Function](https://mlsanity.com/paper/2404.01278/) (2024)
- [EFaR 2023: Efficient Face Recognition Competition](https://mlsanity.com/paper/2308.04168/) (2023)
- [Custom Gradient Estimators are Straight-Through Estimators in Disguise](https://mlsanity.com/paper/2405.05171/) (2024)
- [LILogic Net: Compact Logic Gate Networks with Learnable Connectivity for Efficient Hardware Deployment](https://mlsanity.com/paper/2511.12340/) (2025)
- [MUCM-Net: A Mamba Powered UCM-Net for Skin Lesion Segmentation](https://mlsanity.com/paper/2405.15925/) (2024)
- [A Comprehensive Review and a Taxonomy of Edge Machine Learning: Requirements, Paradigms, and Techniques](https://mlsanity.com/paper/2302.08571/) (2023)
- [Taming Binarized Neural Networks and Mixed-Integer Programs](https://mlsanity.com/paper/2310.04469/) (2023)
- [Efficient Computation of Shap Explanation Scores for Neural Network Classifiers via Knowledge Compilation](https://mlsanity.com/paper/2303.06516/) (2023)
- [Projected Stochastic Gradient Descent with Quantum Annealed Binary Gradients](https://mlsanity.com/paper/2310.15128/) (2023)
- [RepBNN: towards a precise Binary Neural Network with Enhanced Feature Map via Repeating](https://mlsanity.com/paper/2207.09049/) (2022)
- [Straight-Through meets Sparse Recovery: the Support Exploration Algorithm](https://mlsanity.com/paper/2301.13584/) (2023)
- [UCM-Net: A Lightweight and Efficient Solution for Skin Lesion Segmentation using MLP and CNN](https://mlsanity.com/paper/2310.09457/) (2023)
- [HEQuant: Marrying Homomorphic Encryption and Quantization for Communication-Efficient Private Inference](https://mlsanity.com/paper/2401.15970/) (2024)
- [Training Deep Morphological Neural Networks as Universal Approximators](https://mlsanity.com/paper/2505.09710/) (2025)
- [S3NN: Time Step Reduction of Spiking Surrogate Gradients for Training Energy Efficient Single-Step Spiking Neural Networks](https://mlsanity.com/paper/2201.10879/) (2022)
- [How to train accurate BNNs for embedded systems?](https://mlsanity.com/paper/2206.12322/) (2022)
- [Training Neural Networks with Universal Adiabatic Quantum Computing](https://mlsanity.com/paper/2308.13028/) (2023)
- [Trainable Fixed-Point Quantization for Deep Learning Acceleration on FPGAs](https://mlsanity.com/paper/2401.17544/) (2024)
- [Quantized Approximately Orthogonal Recurrent Neural Networks](https://mlsanity.com/paper/2402.04012/) (2024)
- [Verifying Properties of Binary Neural Networks Using Sparse Polynomial Optimization](https://mlsanity.com/paper/2405.17049/) (2024)
- [Training Multi-Layer Binary Neural Networks With Local Binary Error Signals](https://mlsanity.com/paper/2412.00119/) (2024)
- [Quaternion-Hadamard Network: A Novel Defense Against Adversarial Attacks with a New Dataset](https://mlsanity.com/paper/2502.10452/) (2025)
- [HTMA-Net: Towards Multiplication-Avoiding Neural Networks via Hadamard Transform and In-Memory Computing](https://mlsanity.com/paper/2509.23103/) (2025)
- [Expressive Power of Floating-Point Neural Networks with Arbitrary Reduction Orders and Inexact Activation Implementations](https://mlsanity.com/paper/2605.28704/) (2026)

## Abstract (arXiv)

Deep learning (DL) has recently changed the development of intelligent systems and is widely adopted in many real-life applications. Despite their various benefits and potentials, there is a high demand for DL processing in different computationally limited and energy-constrained devices. It is natural to study game-changing technologies such as Binary Neural Networks (BNN) to increase deep learning capabilities. Recently remarkable progress has been made in BNN since they can be implemented and embedded on tiny restricted devices and save a significant amount of storage, computation cost, and energy consumption. However, nearly all BNN acts trade with extra memory, computation cost, and higher performance. This article provides a complete overview of recent developments in BNN. This article focuses exclusively on 1-bit activations and weights 1-bit convolution networks, contrary to previous surveys in which low-bit works are mixed in. It conducted a complete investigation of BNN's development -from their predecessors to the latest BNN algorithms/techniques, presenting a broad design pipeline and discussing each module's variants. Along the way, it examines BNN (a) purpose: their early successes and challenges; (b) BNN optimization: selected representative works that contain essential optimization techniques; (c) deployment: open-source frameworks for BNN modeling and development; (d) terminal: efficient computing architectures and devices for BNN and (e) applications: diverse applications with BNN. Moreover, this paper discusses potential directions and future research opportunities in each section.