# Effective Quantization Approaches for Recurrent Neural Networks

arXiv: https://arxiv.org/abs/1802.02615 · HTML: https://mlsanity.com/paper/1802.02615/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Feb 2018.

## Built on by (most cited first, 10 of 10)

- [NASH: A Simple Unified Framework of Structured Pruning for Accelerating Encoder-Decoder Language Models](https://mlsanity.com/paper/2310.10054/) (2023)
- [Histogram-Equalized Quantization for logic-gated Residual Neural Networks](https://mlsanity.com/paper/2501.04517/) (2025)
- [Spectral Pruning for Recurrent Neural Networks](https://mlsanity.com/paper/2105.10832/) (2021)
- [Vau da muntanialas: Energy-efficient multi-die scalable acceleration of RNN inference](https://mlsanity.com/paper/2202.07462/) (2022)
- [Quantizing Recursive Reasoning Models](https://mlsanity.com/paper/2607.16237/) (2026)
- [Quantized Approximately Orthogonal Recurrent Neural Networks](https://mlsanity.com/paper/2402.04012/) (2024)
- [BILLNET: A Binarized Conv3D-LSTM Network with Logic-gated residual architecture for hardware-efficient video inference](https://mlsanity.com/paper/2501.14495/) (2025)
- [FullPack: Full Vector Utilization for Sub-Byte Quantized Inference on General Purpose CPUs](https://mlsanity.com/paper/2211.06982/) (2022)
- [Towards a tailored mixed-precision sub-8-bit quantization scheme for Gated Recurrent Units using Genetic Algorithms](https://mlsanity.com/paper/2402.12263/) (2024)
- [HadamRNN: Binary and Sparse Ternary Orthogonal RNNs](https://mlsanity.com/paper/2502.00047/) (2025)

## Abstract (arXiv)

Deep learning, and in particular Recurrent Neural Networks (RNN) have shown superior accuracy in a large variety of tasks including machine translation, language understanding, and movie frame generation. However, these deep learning approaches are very expensive in terms of computation. In most cases, Graphic Processing Units (GPUs) are in used for large scale implementations. Meanwhile, energy efficient RNN approaches are proposed for deploying solutions on special purpose hardware including Field Programming Gate Arrays (FPGAs) and mobile platforms. In this paper, we propose an effective quantization approach for Recurrent Neural Networks (RNN) techniques including Long Short Term Memory (LSTM), Gated Recurrent Units (GRU), and Convolutional Long Short Term Memory (ConvLSTM). We have implemented different quantization methods including Binary Connect {-1, 1}, Ternary Connect {-1, 0, 1}, and Quaternary Connect {-1, -0.5, 0.5, 1}. These proposed approaches are evaluated on different datasets for sentiment analysis on IMDB and video frame predictions on the moving MNIST dataset. The experimental results are compared against the full precision versions of the LSTM, GRU, and ConvLSTM. They show promising results for both sentiment analysis and video frame prediction.