# TinyLSTMs: Efficient Neural Speech Enhancement for Hearing Aids

arXiv: https://arxiv.org/abs/2005.11138 · HTML: https://mlsanity.com/paper/2005.11138/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 May 2020.

## Built on by (most cited first, 30 of 42)

- [Machine Learning for Microcontroller-Class Hardware: A Review](https://mlsanity.com/paper/2205.14550/) (2022)
- [Layer Collaboration in the Forward-Forward Algorithm](https://mlsanity.com/paper/2305.12393/) (2023)
- [S2TA: Exploiting Structured Sparsity for Energy-Efficient Mobile CNN Acceleration](https://mlsanity.com/paper/2107.07983/) (2021)
- [Interactive Feature Fusion for End-to-End Noise-Robust Speech Recognition](https://mlsanity.com/paper/2110.05267/) (2021)
- [Real-time Denoising and Dereverberation with Tiny Recurrent U-Net](https://mlsanity.com/paper/2102.03207/) (2021)
- [TinyML: Tools, Applications, Challenges, and Future Research Directions](https://mlsanity.com/paper/2303.13569/) (2023)
- [Latent Space Explanation by Intervention](https://mlsanity.com/paper/2112.04895/) (2021)
- [UDC: Unified DNAS for Compressible TinyML Models](https://mlsanity.com/paper/2201.05842/) (2022)
- [Self-Supervised Learning for Speech Enhancement through Synthesis](https://mlsanity.com/paper/2211.02542/) (2022)
- [Noise-aware Speech Enhancement using Diffusion Probabilistic Model](https://mlsanity.com/paper/2307.08029/) (2023)
- [Two-Step Knowledge Distillation for Tiny Speech Enhancement](https://mlsanity.com/paper/2309.08144/) (2023)
- [Efficient Neural Networks for Tiny Machine Learning: A Comprehensive Review](https://mlsanity.com/paper/2311.11883/) (2023)
- [Weight, Block or Unit? Exploring Sparsity Tradeoffs for Speech Enhancement on Tiny Neural Accelerators](https://mlsanity.com/paper/2111.02351/) (2021)
- [Hybrid Neural Networks for On-device Directional Hearing](https://mlsanity.com/paper/2112.05893/) (2021)
- [Accelerating RNN-based Speech Enhancement on a Multi-Core MCU with Mixed FP16-INT8 Post-Training Quantization](https://mlsanity.com/paper/2210.07692/) (2022)
- [Tiny Machine Learning for Concept Drift](https://mlsanity.com/paper/2107.14759/) (2021)
- [AnalogNets: ML-HW Co-Design of Noise-robust TinyML Models and Always-On Analog Compute-in-Memory Accelerator](https://mlsanity.com/paper/2111.06503/) (2021)
- [Dynamic nsNet2: Efficient Deep Noise Suppression with Early Exiting](https://mlsanity.com/paper/2308.16678/) (2023)
- [Principled Pruning of Bayesian Neural Networks through Variational Free Energy Minimization](https://mlsanity.com/paper/2210.09134/) (2022)
- [Doping: A technique for efficient compression of LSTM models using sparse structured additive matrices](https://mlsanity.com/paper/2102.07071/) (2021)
- [A Functional Information Perspective on Model Interpretation](https://mlsanity.com/paper/2206.05700/) (2022)
- [Dynamic Gated Recurrent Neural Network for Compute-efficient Speech Enhancement](https://mlsanity.com/paper/2408.12425/) (2024)
- [Adaptive Slimming for Scalable and Efficient Speech Enhancement](https://mlsanity.com/paper/2507.04879/) (2025)
- [BLOOM-Net: Blockwise Optimization for Masking Networks Toward Scalable and Efficient Speech Enhancement](https://mlsanity.com/paper/2111.09372/) (2021)
- [Deep neural network techniques for monaural speech enhancement: state of the art analysis](https://mlsanity.com/paper/2212.00369/) (2022)
- [Time-domain Speech Enhancement with Generative Adversarial Learning](https://mlsanity.com/paper/2103.16149/) (2021)
- [End-to-end Keyword Spotting using Neural Architecture Search and Quantization](https://mlsanity.com/paper/2104.06666/) (2021)
- [Resource-Efficient Speech Quality Prediction through Quantization Aware Training and Binary Activation Maps](https://mlsanity.com/paper/2407.04578/) (2024)
- [Towards Sub-millisecond Latency Real-Time Speech Enhancement Models on Hearables](https://mlsanity.com/paper/2409.18239/) (2024)
- [NASTAR: Noise Adaptive Speech Enhancement with Target-Conditional Resampling](https://mlsanity.com/paper/2206.09058/) (2022)

## Abstract (arXiv)

Modern speech enhancement algorithms achieve remarkable noise suppression by means of large recurrent neural networks (RNNs). However, large RNNs limit practical deployment in hearing aid hardware (HW) form-factors, which are battery powered and run on resource-constrained microcontroller units (MCUs) with limited memory capacity and compute capability. In this work, we use model compression techniques to bridge this gap. We define the constraints imposed on the RNN by the HW and describe a method to satisfy them. Although model compression techniques are an active area of research, we are the first to demonstrate their efficacy for RNN speech enhancement, using pruning and integer quantization of weights/activations. We also demonstrate state update skipping, which reduces the computational load. Finally, we conduct a perceptual evaluation of the compressed models to verify audio quality on human raters. Results show a reduction in model size and operations of 11.9× and 2.9×, respectively, over the baseline for compressed models, without a statistical difference in listening preference and only exhibiting a loss of 0.55dB SDR. Our model achieves a computational latency of 2.39ms, well within the 10ms target and 351× better than previous work.