# Bitwise Neural Networks

arXiv: https://arxiv.org/abs/1601.06071 · HTML: https://mlsanity.com/paper/1601.06071/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jan 2016.

## Built on by (most cited first, 29 of 29)

- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classification](https://mlsanity.com/paper/2205.07877/) (2022)
- [PokeBNN: A Binary Pursuit of Lightweight Accuracy](https://mlsanity.com/paper/2112.00133/) (2021)
- [Online Continual Learning for Embedded Devices](https://mlsanity.com/paper/2203.10681/) (2022)
- [A comprehensive review of Binary Neural Network](https://mlsanity.com/paper/2110.06804/) (2021)
- [ZeroQuant-FP: A Leap Forward in LLMs Post-Training W4A8 Quantization Using Floating-Point Formats](https://mlsanity.com/paper/2307.09782/) (2023)
- [Meta-Aggregator: Learning to Aggregate for 1-bit Graph Neural Networks](https://mlsanity.com/paper/2109.12872/) (2021)
- [BiBench: Benchmarking and Analyzing Network Binarization](https://mlsanity.com/paper/2301.11233/) (2023)
- [Extreme Compression for Pre-trained Transformers Made Simple and Efficient](https://mlsanity.com/paper/2206.01859/) (2022)
- ["BNN - BN = ?": Training Binary Neural Networks without Batch Normalization](https://mlsanity.com/paper/2104.08215/) (2021)
- [Sparse PointPillars: Maintaining and Exploiting Input Sparsity to Improve Runtime on Embedded Systems](https://mlsanity.com/paper/2106.06882/) (2021)
- [Resource-Efficient Separation Transformer](https://mlsanity.com/paper/2206.09507/) (2022)
- [Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applications](https://mlsanity.com/paper/2508.15008/) (2025)
- [Stochastic Configuration Machines for Industrial Artificial Intelligence](https://mlsanity.com/paper/2308.13570/) (2023)
- [Efficient and Robust Mixed-Integer Optimization Methods for Training Binarized Deep Neural Networks](https://mlsanity.com/paper/2110.11382/) (2021)
- [Binary Morphological Neural Network](https://mlsanity.com/paper/2203.12337/) (2022)
- [Laplace-HDC: Understanding the geometry of binary hyperdimensional computing](https://mlsanity.com/paper/2404.10759/) (2024)
- [Resource-Efficient Federated Hyperdimensional Computing](https://mlsanity.com/paper/2306.01339/) (2023)
- [AskewSGD : An Annealed interval-constrained Optimisation method to train Quantized Neural Networks](https://mlsanity.com/paper/2211.03741/) (2022)
- [Smoothed Differential Privacy](https://mlsanity.com/paper/2107.01559/) (2021)
- [Low-complexity Approximate Convolutional Neural Networks](https://mlsanity.com/paper/2208.00087/) (2022)
- [Explainable Lifelong Stream Learning Based on "Glocal" Pairwise Fusion](https://mlsanity.com/paper/2306.13410/) (2023)
- [Automated Heterogeneous Low-Bit Quantization of Multi-Model Deep Learning Inference Pipeline](https://mlsanity.com/paper/2311.05870/) (2023)
- [10K is Enough: An Ultra-Lightweight Binarized Network for Infrared Small-Target Detection](https://mlsanity.com/paper/2503.02662/) (2025)
- [TTQ: Activation-Aware Test-Time Quantization to Accelerate LLM Inference On The Fly](https://mlsanity.com/paper/2603.19296/) (2026)
- [Saving RNN Computations with a Neuron-Level Fuzzy Memoization Scheme](https://mlsanity.com/paper/2202.06563/) (2022)
- [Binary Multi Channel Morphological Neural Network](https://mlsanity.com/paper/2204.08768/) (2022)
- [Sparsifying Binary Networks](https://mlsanity.com/paper/2207.04974/) (2022)
- [A foundation for exact binarized morphological neural networks](https://mlsanity.com/paper/2401.03830/) (2024)

## Abstract (arXiv)

Based on the assumption that there exists a neural network that efficiently represents a set of Boolean functions between all binary inputs and outputs, we propose a process for developing and deploying neural networks whose weight parameters, bias terms, input, and intermediate hidden layer output signals, are all binary-valued, and require only basic bit logic for the feedforward pass. The proposed Bitwise Neural Network (BNN) is especially suitable for resource-constrained environments, since it replaces either floating or fixed-point arithmetic with significantly more efficient bitwise operations. Hence, the BNN requires for less spatial complexity, less memory bandwidth, and less power consumption in hardware. In order to design such networks, we propose to add a few training schemes, such as weight compression and noisy backpropagation, which result in a bitwise network that performs almost as well as its corresponding real-valued network. We test the proposed network on the MNIST dataset, represented using binary features, and show that BNNs result in competitive performance while offering dramatic computational savings.