# MOHAQ: Multi-Objective Hardware-Aware Quantization of Recurrent Neural Networks

arXiv: https://arxiv.org/abs/2108.01192 · HTML: https://mlsanity.com/paper/2108.01192/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 55 references that are in the map.
- First posted on arXiv 2 Aug 2021.

## Builds on (most cited first, 23 of 23)

- [Up or Down? Adaptive Rounding for Post-Training Quantization](https://mlsanity.com/paper/2004.10568/) (2020)
- [BinaryConnect: Training Deep Neural Networks with binary weights during propagations](https://mlsanity.com/paper/1511.00363/) (2015)
- [HAQ: Hardware-Aware Automated Quantization with Mixed Precision](https://mlsanity.com/paper/1811.08886/) (2018)
- [ZeroQ: A Novel Zero Shot Quantization Framework](https://mlsanity.com/paper/2001.00281/) (2020)
- [pymoo: Multi-objective Optimization in Python](https://mlsanity.com/paper/2002.04504/) (2020)
- [Improving Neural Network Quantization without Retraining using Outlier Channel Splitting](https://mlsanity.com/paper/1901.09504/) (2019)
- [NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applications](https://mlsanity.com/paper/1804.03230/) (2018)
- [Designing Energy-Efficient Convolutional Neural Networks using Energy-Aware Pruning](https://mlsanity.com/paper/1611.05128/) (2016)
- [Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Networks](https://mlsanity.com/paper/1712.01507/) (2017)
- [Loss Aware Post-training Quantization](https://mlsanity.com/paper/1911.07190/) (2019)
- [Post-Training Piecewise Linear Quantization for Deep Neural Networks](https://mlsanity.com/paper/2002.00104/) (2020)
- [NeST: A Neural Network Synthesis Tool Based on a Grow-and-Prune Paradigm](https://mlsanity.com/paper/1711.02017/) (2017)
- [Light Gated Recurrent Units for Speech Recognition](https://mlsanity.com/paper/1803.10225/) (2018)
- [The PyTorch-Kaldi Speech Recognition Toolkit](https://mlsanity.com/paper/1811.07453/) (2018)
- [Resiliency of Deep Neural Networks under Quantization](https://mlsanity.com/paper/1511.06488/) (2015)
- [Optimizing Speech Recognition For The Edge](https://mlsanity.com/paper/1909.12408/) (2019)
- [DeepIoT: Compressing Deep Neural Network Structures for Sensing Systems with a Compressor-Critic Framework](https://mlsanity.com/paper/1706.01215/) (2017)
- [C-LSTM: Enabling Efficient LSTM using Structured Compression Techniques on FPGAs](https://mlsanity.com/paper/1803.06305/) (2018)
- [FINN-L: Library Extensions and Design Trade-off Analysis for Variable Precision LSTM Networks on FPGAs](https://mlsanity.com/paper/1807.04093/) (2018)
- [Recurrent Neural Networks: An Embedded Computing Perspective](https://mlsanity.com/paper/1908.07062/) (2019)
- [E-RNN: Design Optimization for Efficient Recurrent Neural Networks in FPGAs](https://mlsanity.com/paper/1812.07106/) (2018)
- [Approximate FPGA-based LSTMs under Computation Time Constraints](https://mlsanity.com/paper/1801.02190/) (2018)
- [Neural Machine Translation with 4-Bit Precision and Beyond](https://mlsanity.com/paper/1909.06091/) (2019)

## Abstract (arXiv)

The compression of deep learning models is of fundamental importance in deploying such models to edge devices. The selection of compression parameters can be automated to meet changes in the hardware platform and application using optimization algorithms. This article introduces a Multi-Objective Hardware-Aware Quantization (MOHAQ) method, which considers hardware efficiency and inference error as objectives for mixed-precision quantization. The proposed method feasibly evaluates candidate solutions in a large search space by relying on two steps. First, post-training quantization is applied for fast solution evaluation (inference-only search). Second, we propose the "beacon-based search" to retrain selected solutions only and use them as beacons to know the effect of retraining on other solutions. We use a speech recognition model based on Simple Recurrent Unit (SRU) using the TIMIT dataset and apply our method to run on SiLago and Bitfusion platforms. We provide experimental evaluations showing that SRU can be compressed up to 8x by post-training quantization without any significant error increase. On SiLago, we found solutions that achieve 97% and 86% of the maximum possible speedup and energy saving, with a minor increase in error. On Bitfusion, beacon-based search reduced the error gain of inference-only search by up to 4.9 percentage points.