# Alternating Multi-bit Quantization for Recurrent Neural Networks

arXiv: https://arxiv.org/abs/1802.00150 · HTML: https://mlsanity.com/paper/1802.00150/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Feb 2018.

## Built on by (most cited first, 30 of 31)

- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [Understanding and Overcoming the Challenges of Efficient Transformer Quantization](https://mlsanity.com/paper/2109.12948/) (2021)
- [LUT-GEMM: Quantized Matrix Multiplication based on LUTs for Efficient Inference in Large-Scale Generative Language Models](https://mlsanity.com/paper/2206.09557/) (2022)
- [ARB-LLM: Alternating Refined Binarizations for Large Language Models](https://mlsanity.com/paper/2410.03129/) (2024)
- [ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterization](https://mlsanity.com/paper/2406.05981/) (2024)
- [AlphaTuning: Quantization-Aware Parameter-Efficient Adaptation of Large-Scale Pre-Trained Language Models](https://mlsanity.com/paper/2210.03858/) (2022)
- [Spartus: A 9.4 TOp/s FPGA-based LSTM Accelerator Exploiting Spatio-Temporal Sparsity](https://mlsanity.com/paper/2108.02297/) (2021)
- [Demystifying and Generalizing BinaryConnect](https://mlsanity.com/paper/2110.13220/) (2021)
- [QReg: On Regularization Effects of Quantization](https://mlsanity.com/paper/2206.12372/) (2022)
- [Mixed Precision of Quantization of Transformer Language Models for Speech Recognition](https://mlsanity.com/paper/2112.11540/) (2021)
- [Finding the Task-Optimal Low-Bit Sub-Distribution in Deep Neural Networks](https://mlsanity.com/paper/2112.15139/) (2021)
- [What Do Compressed Multilingual Machine Translation Models Forget?](https://mlsanity.com/paper/2205.10828/) (2022)
- [FIGLUT: An Energy-Efficient Accelerator Design for FP-INT GEMM Using Look-Up Tables](https://mlsanity.com/paper/2503.06862/) (2025)
- [AnyBCQ: Hardware Efficient Flexible Binary-Coded Quantization for Multi-Precision LLMs](https://mlsanity.com/paper/2510.10467/) (2025)
- [4-bit Quantization of LSTM-based Speech Recognition Models](https://mlsanity.com/paper/2108.12074/) (2021)
- [QGen: On the Ability to Generalize in Quantization Aware Training](https://mlsanity.com/paper/2404.11769/) (2024)
- [Unifying Uniform and Binary-coding Quantization for Accurate Compression of Large Language Models](https://mlsanity.com/paper/2506.03781/) (2025)
- [Low-bit Quantization of Recurrent Neural Network Language Models Using Alternating Direction Methods of Multipliers](https://mlsanity.com/paper/2111.14836/) (2021)
- [Vau da muntanialas: Energy-efficient multi-die scalable acceleration of RNN inference](https://mlsanity.com/paper/2202.07462/) (2022)
- [LiftQuant: Continuous Bit-Width LLM via Dimensional Lifting and Projection](https://mlsanity.com/paper/2606.04050/) (2026)
- [Encoding Weights of Irregular Sparsity for Fixed-to-Fixed Model Compression](https://mlsanity.com/paper/2105.01869/) (2021)
- [Quantized Approximately Orthogonal Recurrent Neural Networks](https://mlsanity.com/paper/2402.04012/) (2024)
- [Modulating Regularization Frequency for Efficient Compression-Aware Model Training](https://mlsanity.com/paper/2105.01875/) (2021)
- [Quantized Neural Networks via {-1, +1} Encoding Decomposition and Acceleration](https://mlsanity.com/paper/2106.09886/) (2021)
- [Embedding Compression for Text Classification Using Dictionary Screening](https://mlsanity.com/paper/2211.12715/) (2022)
- [HadamRNN: Binary and Sparse Ternary Orthogonal RNNs](https://mlsanity.com/paper/2502.00047/) (2025)
- [Binary Quadratic Quantization: Beyond First-Order Quantization for Real-Valued Matrix Compression](https://mlsanity.com/paper/2510.18650/) (2025)
- [ELUTQ: Optimizing Quantization Accuracy under LUT-Based Computation for Edge LLMs](https://mlsanity.com/paper/2510.19482/) (2025)
- [Rethinking Output Alignment For 1-bit Post-Training Quantization of Large Language Models](https://mlsanity.com/paper/2512.21651/) (2025)
- [When Quantization Breaks Memory: Recurrent-State Write-Back in Low-Precision Temporal Inference](https://mlsanity.com/paper/2609.04490/) (2026)

## Abstract (arXiv)

Recurrent neural networks have achieved excellent performance in many applications. However, on portable devices with limited resources, the models are often too large to deploy. For applications on the server with large scale concurrent requests, the latency during inference can also be very critical for costly computing resources. In this work, we address these problems by quantizing the network, both weights and activations, into multiple binary codes {-1,+1}. We formulate the quantization as an optimization problem. Under the key observation that once the quantization coefficients are fixed the binary codes can be derived efficiently by binary search tree, alternating minimization is then applied. We test the quantization for two well-known RNNs, i.e., long short term memory (LSTM) and gated recurrent unit (GRU), on the language models. Compared with the full-precision counter part, by 2-bit quantization we can achieve ~16x memory saving and ~6x real inference acceleration on CPUs, with only a reasonable loss in the accuracy. By 3-bit quantization, we can achieve almost no loss in the accuracy or even surpass the original model, with ~10.5x memory saving and ~3x real inference acceleration. Both results beat the exiting quantization works with large margins. We extend our alternating quantization to image classification tasks. In both RNNs and feedforward neural networks, the method also achieves excellent performance.