# Accelerating RNN-based Speech Enhancement on a Multi-Core MCU with Mixed FP16-INT8 Post-Training Quantization

arXiv: https://arxiv.org/abs/2210.07692 · HTML: https://mlsanity.com/paper/2210.07692/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 16 references that are in the map.
- First posted on arXiv 14 Oct 2022.

## Builds on (most cited first, 11 of 11)

- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [Real Time Speech Enhancement in the Waveform Domain](https://mlsanity.com/paper/2006.12847/) (2020)
- [DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancement](https://mlsanity.com/paper/2008.00264/) (2020)
- [A scalable noisy speech dataset and online subjective test framework](https://mlsanity.com/paper/1909.08050/) (2019)
- [Interspeech 2021 Deep Noise Suppression Challenge](https://mlsanity.com/paper/2101.01902/) (2021)
- [TinyLSTMs: Efficient Neural Speech Enhancement for Hearing Aids](https://mlsanity.com/paper/2005.11138/) (2020)
- [A Hybrid DSP/Deep Learning Approach to Real-Time Full-Band Speech Enhancement](https://mlsanity.com/paper/1709.08243/) (2017)
- [A Perceptually-Motivated Approach for Low-Complexity, Real-Time Enhancement of Fullband Speech](https://mlsanity.com/paper/2008.04259/) (2020)
- [Weighted Speech Distortion Losses for Neural-network-based Real-time Speech Enhancement](https://mlsanity.com/paper/2001.10601/) (2020)
- [Weight, Block or Unit? Exploring Sparsity Tradeoffs for Speech Enhancement on Tiny Neural Accelerators](https://mlsanity.com/paper/2111.02351/) (2021)

## Built on by (most cited first, 8 of 8)

- [On-Device Domain Learning for Keyword Spotting on Low-Power Extreme Edge Embedded Systems](https://mlsanity.com/paper/2403.10549/) (2024)
- [Reduced Precision Floating-Point Optimization for Deep Neural Network On-Device Learning on MicroControllers](https://mlsanity.com/paper/2305.19167/) (2023)
- [Dynamic nsNet2: Efficient Deep Noise Suppression with Early Exiting](https://mlsanity.com/paper/2308.16678/) (2023)
- [Adaptive Slimming for Scalable and Efficient Speech Enhancement](https://mlsanity.com/paper/2507.04879/) (2025)
- [When Side-Channel Attacks Break the Black-Box Property of Embedded Artificial Intelligence](https://mlsanity.com/paper/2311.14005/) (2023)
- [TF-MLPNet: Tiny Real-Time Neural Speech Separation](https://mlsanity.com/paper/2508.03047/) (2025)
- [Beyond Model Size: Redesigning LiSenNet for embedded speech enhancement](https://mlsanity.com/paper/2609.29866/) (2026)
- [Transformer-based Neural Beamforming for Real-Time Speech Enhancement on Smart Low-Power Hearable Devices](https://mlsanity.com/paper/2609.33755/) (2026)

## Abstract (arXiv)

This paper presents an optimized methodology to design and deploy Speech Enhancement (SE) algorithms based on Recurrent Neural Networks (RNNs) on a state-of-the-art MicroController Unit (MCU), with 1+8 general-purpose RISC-V cores. To achieve low-latency execution, we propose an optimized software pipeline interleaving parallel computation of LSTM or GRU recurrent blocks, featuring vectorized 8-bit integer (INT8) and 16-bit floating-point (FP16) compute units, with manually-managed memory transfers of model parameters. To ensure minimal accuracy degradation with respect to the full-precision models, we propose a novel FP16-INT8 Mixed-Precision Post-Training Quantization (PTQ) scheme that compresses the recurrent layers to 8-bit while the bit precision of remaining layers is kept to FP16. Experiments are conducted on multiple LSTM and GRU based SE models trained on the Valentini dataset, featuring up to 1.24M parameters. Thanks to the proposed approaches, we speed-up the computation by up to 4x with respect to the lossless FP16 baselines. Differently from a uniform 8-bit quantization that degrades the PESQ score by 0.3 on average, the Mixed-Precision PTQ scheme leads to a low-degradation of only 0.06, while achieving a 1.4-1.7x memory saving. Thanks to this compression, we cut the power cost of the external memory by fitting the large models on the limited on-chip non-volatile memory and we gain a MCU power saving of up to 2.5x by reducing the supply voltage from 0.8V to 0.65V while still matching the real-time constraints. Our design results 10x more energy efficient than state-of-the-art SE solutions deployed on single-core MCUs that make use of smaller models and quantization-aware training.