# Training Binary Multilayer Neural Networks for Image Classification using Expectation Backpropagation

arXiv: https://arxiv.org/abs/1503.03562 · HTML: https://mlsanity.com/paper/1503.03562/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Mar 2015.

## Built on by (most cited first, 6 of 6)

- [Pruning and Quantization for Deep Neural Network Acceleration: A Survey](https://mlsanity.com/paper/2101.09671/) (2021)
- [PokeBNN: A Binary Pursuit of Lightweight Accuracy](https://mlsanity.com/paper/2112.00133/) (2021)
- [Estimator Meets Equilibrium Perspective: A Rectified Straight Through Estimator for Binary Neural Networks Training](https://mlsanity.com/paper/2308.06689/) (2023)
- [Low-Precision Stochastic Gradient Langevin Dynamics](https://mlsanity.com/paper/2206.09909/) (2022)
- [An Encoding Framework for Binarized Images using HyperDimensional Computing](https://mlsanity.com/paper/2312.00454/) (2023)
- [Low-bit Quantization of Recurrent Neural Network Language Models Using Alternating Direction Methods of Multipliers](https://mlsanity.com/paper/2111.14836/) (2021)

## Abstract (arXiv)

Compared to Multilayer Neural Networks with real weights, Binary Multilayer Neural Networks (BMNNs) can be implemented more efficiently on dedicated hardware. BMNNs have been demonstrated to be effective on binary classification tasks with Expectation BackPropagation (EBP) algorithm on high dimensional text datasets. In this paper, we investigate the capability of BMNNs using the EBP algorithm on multiclass image classification tasks. The performances of binary neural networks with multiple hidden layers and different numbers of hidden units are examined on MNIST. We also explore the effectiveness of image spatial filters and the dropout technique in BMNNs. Experimental results on MNIST dataset show that EBP can obtain 2.12% test error with binary weights and 1.66% test error with real weights, which is comparable to the results of standard BackPropagation algorithm on fully connected MNNs.