# Input Layer Binarization with Bit-Plane Encoding

arXiv: https://arxiv.org/abs/2305.02885 · HTML: https://mlsanity.com/paper/2305.02885/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 35 references that are in the map.
- First posted on arXiv 4 May 2023.

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design](https://mlsanity.com/paper/1807.11164/) (2018)
- [EfficientNetV2: Smaller Models and Faster Training](https://mlsanity.com/paper/2104.00298/) (2021)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients](https://mlsanity.com/paper/1606.06160/) (2016)
- [Learning Structured Sparsity in Deep Neural Networks](https://mlsanity.com/paper/1608.03665/) (2016)
- [PACT: Parameterized Clipping Activation for Quantized Neural Networks](https://mlsanity.com/paper/1805.06085/) (2018)
- [LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks](https://mlsanity.com/paper/1807.10029/) (2018)
- [Coordinate Attention for Efficient Mobile Network Design](https://mlsanity.com/paper/2103.02907/) (2021)
- [Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithm](https://mlsanity.com/paper/1808.00278/) (2018)
- [ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functions](https://mlsanity.com/paper/2003.03488/) (2020)
- [Forward and Backward Information Retention for Accurate Binary Neural Networks](https://mlsanity.com/paper/1909.10788/) (2019)
- [Towards Accurate Binary Convolutional Neural Network](https://mlsanity.com/paper/1711.11294/) (2017)
- [Training Binary Neural Networks with Real-to-Binary Convolutions](https://mlsanity.com/paper/2003.11535/) (2020)
- [High-Capacity Expert Binary Networks](https://mlsanity.com/paper/2010.03558/) (2020)
- [MeliusNet: Can Binary Neural Networks Achieve MobileNet-level Accuracy?](https://mlsanity.com/paper/2001.05936/) (2020)
- [FracBNN: Accurate and FPGA-Efficient Binary Neural Networks with Fractional Activations](https://mlsanity.com/paper/2012.12206/) (2020)
- [Projection Convolutional Neural Networks for 1-bit CNNs via Discrete Back Propagation](https://mlsanity.com/paper/1811.12755/) (2018)
- [The High-Dimensional Geometry of Binary Neural Networks](https://mlsanity.com/paper/1705.07199/) (2017)
- [Larq Compute Engine: Design, Benchmark, and Deploy State-of-the-Art Binarized Neural Networks](https://mlsanity.com/paper/2011.09398/) (2020)
- [A Main/Subsidiary Network Framework for Simplifying Binary Neural Network](https://mlsanity.com/paper/1812.04210/) (2018)
- [Binary Input Layer: Training of CNN models with binary input data](https://mlsanity.com/paper/1812.03410/) (2018)
- [RepBNN: towards a precise Binary Neural Network with Enhanced Feature Map via Repeating](https://mlsanity.com/paper/2207.09049/) (2022)

## Built on by (most cited first, 3 of 3)

- [Improvement of Spiking Neural Network with Bit Planes and Color Models](https://mlsanity.com/paper/2410.08229/) (2024)
- [Hybrid Layer-Wise ANN-SNN With Surrogate Spike Encoding-Decoding Structure](https://mlsanity.com/paper/2509.24411/) (2025)
- [End-to-end fully-binarized network design: from Generic Learned Thermometer to Block Pruning](https://mlsanity.com/paper/2505.13462/) (2025)

## Abstract (arXiv)

Binary Neural Networks (BNNs) use 1-bit weights and activations to efficiently execute deep convolutional neural networks on edge devices. Nevertheless, the binarization of the first layer is conventionally excluded, as it leads to a large accuracy loss. The few works addressing the first layer binarization, typically increase the number of input channels to enhance data representation; such data expansion raises the amount of operations needed and it is feasible only on systems with enough computational resources. In this work, we present a new method to binarize the first layer using directly the 8-bit representation of input data; we exploit the standard bit-planes encoding to extract features bit-wise (using depth-wise convolutions); after a re-weighting stage, features are fused again. The resulting model is fully binarized and our first layer binarization approach is model independent. The concept is evaluated on three classification datasets (CIFAR10, SVHN and CIFAR100) for different model architectures (VGG and ResNet) and, the proposed technique outperforms state of the art methods both in accuracy and BMACs reduction.