# VIBNN: Hardware Acceleration of Bayesian Neural Networks

arXiv: https://arxiv.org/abs/1802.00822 · HTML: https://mlsanity.com/paper/1802.00822/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Feb 2018.

## Built on by (most cited first, 20 of 20)

- [Bayesian Learning for Neural Networks: an algorithmic survey](https://mlsanity.com/paper/2211.11865/) (2022)
- [Low-Precision Stochastic Gradient Langevin Dynamics](https://mlsanity.com/paper/2206.09909/) (2022)
- [Bayesian continual learning and forgetting in neural networks](https://mlsanity.com/paper/2504.13569/) (2025)
- [On the Effects of Quantisation on Model Uncertainty in Bayesian Neural Networks](https://mlsanity.com/paper/2102.11062/) (2021)
- [High-Performance FPGA-based Accelerator for Bayesian Neural Networks](https://mlsanity.com/paper/2105.09163/) (2021)
- [Spatial-SpinDrop: Spatial Dropout-based Binary Bayesian Neural Network with Spintronics Implementation](https://mlsanity.com/paper/2306.10185/) (2023)
- [MC-CIM: Compute-in-Memory with Monte-Carlo Dropouts for Bayesian Edge Intelligence](https://mlsanity.com/paper/2111.07125/) (2021)
- [Scale-Dropout: Estimating Uncertainty in Deep Neural Networks Using Stochastic Scale](https://mlsanity.com/paper/2311.15816/) (2023)
- [When Monte-Carlo Dropout Meets Multi-Exit: Optimizing Bayesian Neural Networks on FPGA](https://mlsanity.com/paper/2308.06849/) (2023)
- [Optimizing Bayesian Recurrent Neural Networks on an FPGA-based Accelerator](https://mlsanity.com/paper/2106.06048/) (2021)
- [Shift-BNN: Highly-Efficient Probabilistic Bayesian Neural Network Training via Memory-Friendly Pattern Retrieving](https://mlsanity.com/paper/2110.03553/) (2021)
- [Bayesian Inference on Binary Spiking Networks Leveraging Nanoscale Device Stochasticity](https://mlsanity.com/paper/2302.01302/) (2023)
- [Analog Bayesian neural networks are insensitive to the shape of the weight distribution](https://mlsanity.com/paper/2501.05564/) (2025)
- [NeuSpin: Design of a Reliable Edge Neuromorphic System Based on Spintronics for Green AI](https://mlsanity.com/paper/2401.06195/) (2024)
- [Enhancing Dropout-based Bayesian Neural Networks with Multi-Exit on FPGA](https://mlsanity.com/paper/2406.14593/) (2024)
- [Hardware-Aware Neural Dropout Search for Reliable Uncertainty Prediction on FPGA](https://mlsanity.com/paper/2406.16198/) (2024)
- [Accelerating MRI Uncertainty Estimation with Mask-based Bayesian Neural Network](https://mlsanity.com/paper/2407.05521/) (2024)
- [Bayes2IMC: In-Memory Computing for Bayesian Binary Neural Networks](https://mlsanity.com/paper/2411.07902/) (2024)
- [Accelerated Execution of Bayesian Neural Networks using a Single Probabilistic Forward Pass and Code Generation](https://mlsanity.com/paper/2511.23440/) (2025)
- [Algorithm and Hardware Co-Design for Efficient Complex-Valued Uncertainty Estimation](https://mlsanity.com/paper/2604.19993/) (2026)

## Abstract (arXiv)

Bayesian Neural Networks (BNNs) have been proposed to address the problem of model uncertainty in training and inference. By introducing weights associated with conditioned probability distributions, BNNs are capable of resolving the overfitting issue commonly seen in conventional neural networks and allow for small-data training, through the variational inference process. Frequent usage of Gaussian random variables in this process requires a properly optimized Gaussian Random Number Generator (GRNG). The high hardware cost of conventional GRNG makes the hardware implementation of BNNs challenging. In this paper, we propose VIBNN, an FPGA-based hardware accelerator design for variational inference on BNNs. We explore the design space for massive amount of Gaussian variable sampling tasks in BNNs. Specifically, we introduce two high performance Gaussian (pseudo) random number generators: the RAM-based Linear Feedback Gaussian Random Number Generator (RLF-GRNG), which is inspired by the properties of binomial distribution and linear feedback logics; and the Bayesian Neural Network-oriented Wallace Gaussian Random Number Generator. To achieve high scalability and efficient memory access, we propose a deep pipelined accelerator architecture with fast execution and good hardware utilization. Experimental results demonstrate that the proposed VIBNN implementations on an FPGA can achieve throughput of 321,543.4 Images/s and energy efficiency upto 52,694.8 Images/J while maintaining similar accuracy as its software counterpart.