# High-Performance FPGA-based Accelerator for Bayesian Neural Networks

arXiv: https://arxiv.org/abs/2105.09163 · HTML: https://mlsanity.com/paper/2105.09163/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 21 references that are in the map.
- First posted on arXiv 12 May 2021.

## Builds on (most cited first, 11 of 11)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks](https://mlsanity.com/paper/2002.10118/) (2020)
- [Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding](https://mlsanity.com/paper/1511.02680/) (2015)
- [VIBNN: Hardware Acceleration of Bayesian Neural Networks](https://mlsanity.com/paper/1802.00822/) (2018)
- [Stochastic-YOLO: Efficient Probabilistic Object Detection under Dataset Shifts](https://mlsanity.com/paper/2009.02967/) (2020)
- [VINNAS: Variational Inference-based Neural Network Architecture Search](https://mlsanity.com/paper/2007.06103/) (2020)

## Built on by (most cited first, 8 of 8)

- [Scale-Dropout: Estimating Uncertainty in Deep Neural Networks Using Stochastic Scale](https://mlsanity.com/paper/2311.15816/) (2023)
- [When Monte-Carlo Dropout Meets Multi-Exit: Optimizing Bayesian Neural Networks on FPGA](https://mlsanity.com/paper/2308.06849/) (2023)
- [Optimizing Bayesian Recurrent Neural Networks on an FPGA-based Accelerator](https://mlsanity.com/paper/2106.06048/) (2021)
- [Enhancing Dropout-based Bayesian Neural Networks with Multi-Exit on FPGA](https://mlsanity.com/paper/2406.14593/) (2024)
- [Hardware-Aware Neural Dropout Search for Reliable Uncertainty Prediction on FPGA](https://mlsanity.com/paper/2406.16198/) (2024)
- [Accelerating MRI Uncertainty Estimation with Mask-based Bayesian Neural Network](https://mlsanity.com/paper/2407.05521/) (2024)
- [Bayes2IMC: In-Memory Computing for Bayesian Binary Neural Networks](https://mlsanity.com/paper/2411.07902/) (2024)
- [Algorithm and Hardware Co-Design for Efficient Complex-Valued Uncertainty Estimation](https://mlsanity.com/paper/2604.19993/) (2026)

## Abstract (arXiv)

Neural networks (NNs) have demonstrated their potential in a wide range of applications such as image recognition, decision making or recommendation systems. However, standard NNs are unable to capture their model uncertainty which is crucial for many safety-critical applications including healthcare and autonomous vehicles. In comparison, Bayesian neural networks (BNNs) are able to express uncertainty in their prediction via a mathematical grounding. Nevertheless, BNNs have not been as widely used in industrial practice, mainly because of their expensive computational cost and limited hardware performance. This work proposes a novel FPGA-based hardware architecture to accelerate BNNs inferred through Monte Carlo Dropout. Compared with other state-of-the-art BNN accelerators, the proposed accelerator can achieve up to 4 times higher energy efficiency and 9 times better compute efficiency. Considering partial Bayesian inference, an automatic framework is proposed, which explores the trade-off between hardware and algorithmic performance. Extensive experiments are conducted to demonstrate that our proposed framework can effectively find the optimal points in the design space.