mlsanityData to 14 Oct 2026

Optimizing Bayesian Recurrent Neural Networks on an FPGA-based Accelerator

arXiv 2106.06048 ↗First posted 4 Jun 2021cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Neural networks have demonstrated their outstanding performance in a wide range of tasks. Specifically recurrent architectures based on long-short term memory (LSTM) cells have manifested excellent capability to model time dependencies in real-world data. However, standard recurrent architectures cannot estimate their uncertainty which is essential for safety-critical applications such as in medicine. In contrast, Bayesian recurrent neural networks (RNNs) are able to provide uncertainty estimation with improved accuracy. Nonetheless, Bayesian RNNs are computationally and memory demanding, which limits their practicality despite their advantages. To address this issue, we propose an FPGA-based hardware design to accelerate Bayesian LSTM-based RNNs. To further improve the overall algorithmic-hardware performance, a co-design framework is proposed to explore the most fitting algorithmic-hardware configurations for Bayesian RNNs. We conduct extensive experiments on healthcare applications to demonstrate the improvement of our design and the effectiveness of our framework. Compared with GPU implementation, our FPGA-based design can achieve up to 10 times speedup with nearly 106 times higher energy efficiency. To the best of our knowledge, this is the first work targeting acceleration of Bayesian RNNs on FPGAs.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 18 of its 38 references that are in the map. First posted on arXiv 4 Jun 2021.

Builds on

18 of 38 references.

References · 18

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  3. 2013Auto-Encoding Variational Bayescited by 4,053
  4. 2015Unsupervised Learning of Video Representations using LSTMscited by 435
  5. 2015Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understandingcited by 218
  6. 2015A Theoretically Grounded Application of Dropout in Recurrent Neural Networkscited by 161
  7. 2018Fast inference of deep neural networks in FPGAs for particle physicscited by 133
  8. 2016ESE: Efficient Speech Recognition Engine with Sparse LSTM on FPGAcited by 52
  9. 2017Bayesian Recurrent Neural Networkscited by 39
  10. 2018VIBNN: Hardware Acceleration of Bayesian Neural Networkscited by 20
  11. 2017Bayesian Recurrent Neural Network Models for Forecasting and Quantifying Uncertainty in Spatial-Temporal Datacited by 17
  12. 2019Fault Detection and Identification using Bayesian Recurrent Neural Networkscited by 11
6 more
  1. 2018FINN-L: Library Extensions and Design Trade-off Analysis for Variable Precision LSTM Networks on FPGAscited by 10
  2. 2020Efficient Computation Reduction in Bayesian Neural Networks Through Feature Decomposition and Memorizationcited by 10
  3. 2015Recurrent Neural Networks Hardware Implementation on FPGAcited by 8
  4. 2021High-Performance FPGA-based Accelerator for Bayesian Neural Networkscited by 8
  5. 2019Unrolling Ternary Neural Networkscited by 7
  6. 2017Bayesian LSTMs in medicinecited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 0; 2026: 1.

3 papers.

  1. 2024Enhancing Dropout-based Bayesian Neural Networks with Multi-Exit on FPGAcited by 0
  2. 2024Accelerating MRI Uncertainty Estimation with Mask-based Bayesian Neural Networkcited by 0
  3. 2026Algorithm and Hardware Co-Design for Efficient Complex-Valued Uncertainty Estimationcited by 0