# Nonlocal optimization of binary neural networks

arXiv: https://arxiv.org/abs/2204.01935 · HTML: https://mlsanity.com/paper/2204.01935/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 24 references that are in the map.
- First posted on arXiv 5 Apr 2022.

## Builds on (most cited first, 8 of 8)

- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Bayesian Compression for Deep Learning](https://mlsanity.com/paper/1705.08665/) (2017)
- [Soft Weight-Sharing for Neural Network Compression](https://mlsanity.com/paper/1702.04008/) (2017)
- [Loopy Belief Propagation for Approximate Inference: An Empirical Study](https://mlsanity.com/paper/1301.6725/) (2013)
- [Learning Discrete Weights Using the Local Reparameterization Trick](https://mlsanity.com/paper/1710.07739/) (2017)
- [Probabilistic Binary Neural Networks](https://mlsanity.com/paper/1809.03368/) (2018)
- [Revisiting Algebra and Complexity of Inference in Graphical Models](https://mlsanity.com/paper/1409.7410/) (2014)
- [Relaxed Survey Propagation for The Weighted Maximum Satisfiability Problem](https://mlsanity.com/paper/1401.3483/) (2014)

## Abstract (arXiv)

We explore training Binary Neural Networks (BNNs) as a discrete variable inference problem over a factor graph. We study the behaviour of this conversion in an under-parameterized BNN setting and propose stochastic versions of Belief Propagation (BP) and Survey Propagation (SP) message passing algorithms to overcome the intractability of their current formulation. Compared to traditional gradient methods for BNNs, our results indicate that both stochastic BP and SP find better configurations of the parameters in the BNN.