# Recurrent Neural Networks Hardware Implementation on FPGA

arXiv: https://arxiv.org/abs/1511.05552 · HTML: https://mlsanity.com/paper/1511.05552/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Nov 2015.

## Built on by (most cited first, 8 of 8)

- [HMT: Hierarchical Memory Transformer for Efficient Long Context Language Processing](https://mlsanity.com/paper/2405.06067/) (2024)
- [Ultra-low latency recurrent neural network inference on FPGAs for physics applications with hls4ml](https://mlsanity.com/paper/2207.00559/) (2022)
- [Optimizing Bayesian Recurrent Neural Networks on an FPGA-based Accelerator](https://mlsanity.com/paper/2106.06048/) (2021)
- [Real-time Hyper-Dimensional Reconfiguration at the Edge using Hardware Accelerators](https://mlsanity.com/paper/2206.05128/) (2022)
- [A Survey of Distributed Learning in Cloud, Mobile, and Edge Settings](https://mlsanity.com/paper/2405.15079/) (2024)
- [Implementing Neural Network-Based Equalizers in a Coherent Optical Transmission System Using Field-Programmable Gate Arrays](https://mlsanity.com/paper/2212.04703/) (2022)
- [Improving the Performance and Learning Stability of Parallelizable RNNs Designed for Ultra-Low Power Applications](https://mlsanity.com/paper/2605.11855/) (2026)
- [Hardware-Software Co-Design of Scalable, Energy-Efficient Analog Recurrent Computations](https://mlsanity.com/paper/2605.15216/) (2026)

## Abstract (arXiv)

Recurrent Neural Networks (RNNs) have the ability to retain memory and learn data sequences. Due to the recurrent nature of RNNs, it is sometimes hard to parallelize all its computations on conventional hardware. CPUs do not currently offer large parallelism, while GPUs offer limited parallelism due to sequential components of RNN models. In this paper we present a hardware implementation of Long-Short Term Memory (LSTM) recurrent network on the programmable logic Zynq 7020 FPGA from Xilinx. We implemented a RNN with 2 layers and 128 hidden units in hardware and it has been tested using a character level language model. The implementation is more than 21× faster than the ARM CPU embedded on the Zynq 7020 FPGA. This work can potentially evolve to a RNN co-processor for future mobile devices.