# On the quantization of recurrent neural networks

arXiv: https://arxiv.org/abs/2101.05453 · HTML: https://mlsanity.com/paper/2101.05453/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 26 references that are in the map.
- First posted on arXiv 14 Jan 2021.

## Builds on (most cited first, 14 of 14)

- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [LSTM: A Search Space Odyssey](https://mlsanity.com/paper/1503.04069/) (2015)
- [Streaming End-to-end Speech Recognition For Mobile Devices](https://mlsanity.com/paper/1811.06621/) (2018)
- [A Streaming On-Device End-to-End Model Surpassing Server-Side Conventional Model Quality and Latency](https://mlsanity.com/paper/2003.12710/) (2020)
- [Simple Recurrent Units for Highly Parallelizable Recurrence](https://mlsanity.com/paper/1709.02755/) (2017)
- [Optimizing Speech Recognition For The Edge](https://mlsanity.com/paper/1909.12408/) (2019)
- [Personalized Speech recognition on mobile devices](https://mlsanity.com/paper/1603.03185/) (2016)
- [A Cascade Architecture for Keyword Spotting on Mobile Devices](https://mlsanity.com/paper/1712.03603/) (2017)
- [On the efficient representation and execution of deep acoustic models](https://mlsanity.com/paper/1607.04683/) (2016)

## Built on by (most cited first, 7 of 7)

- [TinyML: Tools, Applications, Challenges, and Future Research Directions](https://mlsanity.com/paper/2303.13569/) (2023)
- [Q-S5: Towards Quantized State Space Models](https://mlsanity.com/paper/2406.09477/) (2024)
- [ML-EXray: Visibility into ML Deployment on the Edge](https://mlsanity.com/paper/2111.04779/) (2021)
- [Accelerating Linear Recurrent Neural Networks for the Edge with Unstructured Sparsity](https://mlsanity.com/paper/2502.01330/) (2025)
- [Engineering the Neural Automatic Passenger Counter](https://mlsanity.com/paper/2203.01156/) (2022)
- [Accelerator-Aware Training for Transducer-Based Speech Recognition](https://mlsanity.com/paper/2305.07778/) (2023)
- [Beyond Model Size: Redesigning LiSenNet for embedded speech enhancement](https://mlsanity.com/paper/2609.29866/) (2026)

## Abstract (arXiv)

Integer quantization of neural networks can be defined as the approximation of the high precision computation of the canonical neural network formulation, using reduced integer precision. It plays a significant role in the efficient deployment and execution of machine learning (ML) systems, reducing memory consumption and leveraging typically faster computations. In this work, we present an integer-only quantization strategy for Long Short-Term Memory (LSTM) neural network topologies, which themselves are the foundation of many production ML systems. Our quantization strategy is accurate (e.g. works well with quantization post-training), efficient and fast to execute (utilizing 8 bit integer weights and mostly 8 bit activations), and is able to target a variety of hardware (by leveraging instructions sets available in common CPU architectures, as well as available neural accelerators).