# Training Dynamical Binary Neural Networks with Equilibrium Propagation

arXiv: https://arxiv.org/abs/2103.08953 · HTML: https://mlsanity.com/paper/2103.08953/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 47 references that are in the map.
- First posted on arXiv 16 Mar 2021.

## Builds on (most cited first, 18 of 18)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [BinaryConnect: Training Deep Neural Networks with binary weights during propagations](https://mlsanity.com/paper/1511.00363/) (2015)
- [Quantifying the Carbon Emissions of Machine Learning](https://mlsanity.com/paper/1910.09700/) (2019)
- [Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagation](https://mlsanity.com/paper/1602.05179/) (2016)
- [Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architectures](https://mlsanity.com/paper/1807.04587/) (2018)
- [XNOR-Net++: Improved Binary Neural Networks](https://mlsanity.com/paper/1909.13863/) (2019)
- [Scaling Equilibrium Propagation to Deep ConvNets by Drastically Reducing its Gradient Estimator Bias](https://mlsanity.com/paper/2006.03824/) (2020)
- [Training End-to-End Analog Neural Networks with Equilibrium Propagation](https://mlsanity.com/paper/2006.01981/) (2020)
- [Latent Weights Do Not Exist: Rethinking Binarized Neural Network Optimization](https://mlsanity.com/paper/1906.02107/) (2019)
- [Updates of Equilibrium Prop Match Gradients of Backprop Through Time in an RNN with Static Input](https://mlsanity.com/paper/1905.13633/) (2019)
- [Equivalence of Equilibrium Propagation and Recurrent Backpropagation](https://mlsanity.com/paper/1711.08416/) (2017)
- [EqSpike: Spike-driven Equilibrium Propagation for Neuromorphic Implementations](https://mlsanity.com/paper/2010.07859/) (2020)
- [Equilibrium Propagation with Continual Weight Updates](https://mlsanity.com/paper/2005.04168/) (2020)
- [Large-Scale Neuromorphic Spiking Array Processors: A quest to mimic the brain](https://mlsanity.com/paper/1805.08932/) (2018)
- [Batch Normalized Recurrent Neural Networks](https://mlsanity.com/paper/1510.01378/) (2015)
- [Towards deep learning with spiking neurons in energy based models with contrastive Hebbian plasticity](https://mlsanity.com/paper/1612.03214/) (2016)

## Built on by (most cited first, 13 of 13)

- [A comprehensive review of Binary Neural Network](https://mlsanity.com/paper/2110.06804/) (2021)
- [Energy-based learning algorithms for analog computing: a comparative study](https://mlsanity.com/paper/2312.15103/) (2023)
- [Brain-inspired Computational Intelligence via Predictive Coding](https://mlsanity.com/paper/2308.07870/) (2023)
- [Training an Ising Machine with Equilibrium Propagation](https://mlsanity.com/paper/2305.18321/) (2023)
- [Spike time displacement based error backpropagation in convolutional spiking neural networks](https://mlsanity.com/paper/2108.13621/) (2021)
- [A Review of Neuroscience-Inspired Machine Learning](https://mlsanity.com/paper/2403.18929/) (2024)
- [Training Multi-layer Neural Networks on Ising Machine](https://mlsanity.com/paper/2311.03408/) (2023)
- [A foundation for exact binarized morphological neural networks](https://mlsanity.com/paper/2401.03830/) (2024)
- [CycleBNN: Cyclic Precision Training in Binary Neural Networks](https://mlsanity.com/paper/2410.00050/) (2024)
- [Scalable Equilibrium Propagation via Intermediate Error Signals for Deep Convolutional CRNNs](https://mlsanity.com/paper/2508.15989/) (2025)
- [Hybridizing Equilibrium Propagation with Ising Machines for Efficient Energy-Based Learning](https://mlsanity.com/paper/2606.09112/) (2026)
- [Optimizing Energy-based Neural Network Training with Coherent Ising Machine](https://mlsanity.com/paper/2606.09117/) (2026)
- [Optical Implementation of Equilibrium Propagation Using Spatial Photonic Ising Machines](https://mlsanity.com/paper/2606.13454/) (2026)

## Abstract (arXiv)

Equilibrium Propagation (EP) is an algorithm intrinsically adapted to the training of physical networks, thanks to the local updates of weights given by the internal dynamics of the system. However, the construction of such a hardware requires to make the algorithm compatible with existing neuromorphic CMOS technologies, which generally exploit digital communication between neurons and offer a limited amount of local memory. In this work, we demonstrate that EP can train dynamical networks with binary activations and weights. We first train systems with binary weights and full-precision activations, achieving an accuracy equivalent to that of full-precision models trained by standard EP on MNIST, and losing only 1.9% accuracy on CIFAR-10 with equal architecture. We then extend our method to the training of models with binary activations and weights on MNIST, achieving an accuracy within 1% of the full-precision reference for fully connected architectures and reaching the full-precision accuracy for convolutional architectures. Our extension of EP to binary networks opens new solutions for on-chip learning and provides a compact framework for training BNNs end-to-end with the same circuitry as for inference.