# When Monte-Carlo Dropout Meets Multi-Exit: Optimizing Bayesian Neural Networks on FPGA

arXiv: https://arxiv.org/abs/2308.06849 · HTML: https://mlsanity.com/paper/2308.06849/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 28 references that are in the map.
- First posted on arXiv 13 Aug 2023.

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Deep Ensembles: A Loss Landscape Perspective](https://mlsanity.com/paper/1912.02757/) (2019)
- [Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding](https://mlsanity.com/paper/1511.02680/) (2015)
- [hls4ml: An Open-Source Codesign Workflow to Empower Scientific Low-Power Machine Learning Devices](https://mlsanity.com/paper/2103.05579/) (2021)
- [Adaptable Butterfly Accelerator for Attention-based NNs via Hardware and Algorithm Co-design](https://mlsanity.com/paper/2209.09570/) (2022)
- [VIBNN: Hardware Acceleration of Bayesian Neural Networks](https://mlsanity.com/paper/1802.00822/) (2018)
- [High-Performance FPGA-based Accelerator for Bayesian Neural Networks](https://mlsanity.com/paper/2105.09163/) (2021)
- [Stochastic-Shield: A Probabilistic Approach Towards Training-Free Adversarial Defense in Quantized CNNs](https://mlsanity.com/paper/2105.06512/) (2021)
- [Students are the Best Teacher: Exit-Ensemble Distillation with Multi-Exits](https://mlsanity.com/paper/2104.00299/) (2021)
- [Shift-BNN: Highly-Efficient Probabilistic Bayesian Neural Network Training via Memory-Friendly Pattern Retrieving](https://mlsanity.com/paper/2110.03553/) (2021)

## Built on by (most cited first, 4 of 4)

- [Hardware-Aware Neural Dropout Search for Reliable Uncertainty Prediction on FPGA](https://mlsanity.com/paper/2406.16198/) (2024)
- [Accelerating MRI Uncertainty Estimation with Mask-based Bayesian Neural Network](https://mlsanity.com/paper/2407.05521/) (2024)
- [Enhancing Trustworthiness with Mixed Precision: Benchmarks, Opportunities, and Challenges](https://mlsanity.com/paper/2511.22483/) (2025)
- [Algorithm and Hardware Co-Design for Efficient Complex-Valued Uncertainty Estimation](https://mlsanity.com/paper/2604.19993/) (2026)

## Abstract (arXiv)

Bayesian Neural Networks (BayesNNs) have demonstrated their capability of providing calibrated prediction for safety-critical applications such as medical imaging and autonomous driving. However, the high algorithmic complexity and the poor hardware performance of BayesNNs hinder their deployment in real-life applications. To bridge this gap, this paper proposes a novel multi-exit Monte-Carlo Dropout (MCD)-based BayesNN that achieves well-calibrated predictions with low algorithmic complexity. To further reduce the barrier to adopting BayesNNs, we propose a transformation framework that can generate FPGA-based accelerators for multi-exit MCD-based BayesNNs. Several novel optimization techniques are introduced to improve hardware performance. Our experiments demonstrate that our auto-generated accelerator achieves higher energy efficiency than CPU, GPU, and other state-of-the-art hardware implementations.