# Effective Quantization Methods for Recurrent Neural Networks

arXiv: https://arxiv.org/abs/1611.10176 · HTML: https://mlsanity.com/paper/1611.10176/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Nov 2016.

## Built on by (most cited first, 14 of 14)

- [A Survey on Green Deep Learning](https://mlsanity.com/paper/2111.05193/) (2021)
- [BSQ: Exploring Bit-Level Sparsity for Mixed-Precision Neural Network Quantization](https://mlsanity.com/paper/2102.10462/) (2021)
- [CoMERA: Computing- and Memory-Efficient Training via Rank-Adaptive Tensor Optimization](https://mlsanity.com/paper/2405.14377/) (2024)
- [4-bit Quantization of LSTM-based Speech Recognition Models](https://mlsanity.com/paper/2108.12074/) (2021)
- [NASH: A Simple Unified Framework of Structured Pruning for Accelerating Encoder-Decoder Language Models](https://mlsanity.com/paper/2310.10054/) (2023)
- [Training Integer-Only Deep Recurrent Neural Networks](https://mlsanity.com/paper/2212.11791/) (2022)
- [Histogram-Equalized Quantization for logic-gated Residual Neural Networks](https://mlsanity.com/paper/2501.04517/) (2025)
- [iRNN: Integer-only Recurrent Neural Network](https://mlsanity.com/paper/2109.09828/) (2021)
- [Vau da muntanialas: Energy-efficient multi-die scalable acceleration of RNN inference](https://mlsanity.com/paper/2202.07462/) (2022)
- [Quantizing Recursive Reasoning Models](https://mlsanity.com/paper/2607.16237/) (2026)
- [AutoQNN: An End-to-End Framework for Automatically Quantizing Neural Networks](https://mlsanity.com/paper/2304.03782/) (2023)
- [HadamRNN: Binary and Sparse Ternary Orthogonal RNNs](https://mlsanity.com/paper/2502.00047/) (2025)
- [Dynamic Clustering for Personalized Federated Learning on Heterogeneous Edge Devices](https://mlsanity.com/paper/2508.01580/) (2025)
- [Downsized and Compromised?: Assessing the Faithfulness of Model Compression](https://mlsanity.com/paper/2510.06125/) (2025)

## Abstract (arXiv)

Reducing bit-widths of weights, activations, and gradients of a Neural Network can shrink its storage size and memory usage, and also allow for faster training and inference by exploiting bitwise operations. However, previous attempts for quantization of RNNs show considerable performance degradation when using low bit-width weights and activations. In this paper, we propose methods to quantize the structure of gates and interlinks in LSTM and GRU cells. In addition, we propose balanced quantization methods for weights to further reduce performance degradation. Experiments on PTB and IMDB datasets confirm effectiveness of our methods as performances of our models match or surpass the previous state-of-the-art of quantized RNN.