# Bayes2IMC: In-Memory Computing for Bayesian Binary Neural Networks

arXiv: https://arxiv.org/abs/2411.07902 · HTML: https://mlsanity.com/paper/2411.07902/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 59 references that are in the map.
- First posted on arXiv 12 Nov 2024.

## Builds on (most cited first, 20 of 20)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [BinaryConnect: Training Deep Neural Networks with binary weights during propagations](https://mlsanity.com/paper/1511.00363/) (2015)
- [Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users](https://mlsanity.com/paper/2007.06823/) (2020)
- [The Bayesian Learning Rule](https://mlsanity.com/paper/2107.04562/) (2021)
- [The Case for Bayesian Deep Learning](https://mlsanity.com/paper/2001.10995/) (2020)
- [How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review](https://mlsanity.com/paper/2107.12045/) (2021)
- [Training Binary Neural Networks using the Bayesian Learning Rule](https://mlsanity.com/paper/2002.10778/) (2020)
- [Unlabelled Data Improves Bayesian Uncertainty Calibration under Covariate Shift](https://mlsanity.com/paper/2006.14988/) (2020)
- [VIBNN: Hardware Acceleration of Bayesian Neural Networks](https://mlsanity.com/paper/1802.00822/) (2018)
- [Wide Neural Networks with Bottlenecks are Deep Gaussian Processes](https://mlsanity.com/paper/2001.00921/) (2020)
- [Efficient Computation Reduction in Bayesian Neural Networks Through Feature Decomposition and Memorization](https://mlsanity.com/paper/2005.03857/) (2020)
- [Dark Memory and Accelerator-Rich System Optimization in the Dark Silicon Era](https://mlsanity.com/paper/1602.04183/) (2016)
- [BiSNN: Training Spiking Neural Networks with Binary Weights via Bayesian Learning](https://mlsanity.com/paper/2012.08300/) (2020)
- [High-Performance FPGA-based Accelerator for Bayesian Neural Networks](https://mlsanity.com/paper/2105.09163/) (2021)
- [Calibrating Bayesian Learning via Regularization, Confidence Minimization, and Selective Inference](https://mlsanity.com/paper/2404.11350/) (2024)
- [Bayesian Inference on Binary Spiking Networks Leveraging Nanoscale Device Stochasticity](https://mlsanity.com/paper/2302.01302/) (2023)
- [Bayesian Inference Accelerator for Spiking Neural Networks](https://mlsanity.com/paper/2401.15453/) (2024)
- [Hybrid In-memory Computing Architecture for the Training of Deep Neural Networks](https://mlsanity.com/paper/2102.05271/) (2021)

## Abstract (arXiv)

Bayesian Neural Networks (BNNs) provide superior estimates of uncertainty by generating an ensemble of predictive distributions. However, inference via ensembling is resource-intensive, requiring additional entropy sources to generate stochasticity which increases resource consumption. We introduce Bayes2IMC, an in-memory computing (IMC) architecture designed for binary Bayesian neural networks that leverage nanoscale device stochasticity to generate desired distributions. Our novel approach utilizes Phase-Change Memory (PCM) to harness inherent noise characteristics, enabling the creation of a binary neural network. This design eliminates the necessity for a pre-neuron Analog-to-Digital Converter (ADC), significantly improving power and area efficiency. We also develop a hardware-software co-optimized correction method applied solely on the logits in the final layer to reduce device-induced accuracy variations across deployments on hardware. Additionally, we devise a simple compensation technique that ensures no drop in classification accuracy despite conductance drift of PCM. We validate the effectiveness of our approach on the CIFAR-10 dataset with a VGGBinaryConnect model, achieving accuracy metrics comparable to ideal software implementations as well as results reported in the literature using other technologies. Finally, we present a complete core architecture and compare its projected power, performance, and area efficiency against an equivalent SRAM baseline, showing a 3.8 to 9.6 × improvement in total efficiency (in GOPS/W/mm2) and a 2.2  to 5.6 × improvement in power efficiency (in GOPS/W). In addition, the projected hardware performance of Bayes2IMC surpasses that of most of the BNN architectures based on memristive devices reported in the literature, and achieves up to 20% higher power efficiency compared to the state-of-the-art.