# aTENNuate: Optimized Real-time Speech Enhancement with Deep SSMs on Raw Audio

arXiv: https://arxiv.org/abs/2409.03377 · HTML: https://mlsanity.com/paper/2409.03377/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 34 references that are in the map.
- First posted on arXiv 5 Sep 2024.

## Builds on (most cited first, 28 of 28)

- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Simplified State Space Layers for Sequence Modeling](https://mlsanity.com/paper/2208.04933/) (2022)
- [On the Parameterization and Initialization of Diagonal State Space Models](https://mlsanity.com/paper/2206.11893/) (2022)
- [Real Time Speech Enhancement in the Waveform Domain](https://mlsanity.com/paper/2006.12847/) (2020)
- [It's Raw! Audio Generation with State-Space Models](https://mlsanity.com/paper/2202.09729/) (2022)
- [SEGAN: Speech Enhancement Generative Adversarial Network](https://mlsanity.com/paper/1703.09452/) (2017)
- [DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancement](https://mlsanity.com/paper/2008.00264/) (2020)
- [The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Testing Framework, and Challenge Results](https://mlsanity.com/paper/2005.13981/) (2020)
- [Phase-aware Speech Enhancement with Deep Complex U-Net](https://mlsanity.com/paper/1903.03107/) (2019)
- [An Investigation of Incorporating Mamba for Speech Enhancement](https://mlsanity.com/paper/2405.06573/) (2024)
- [Mamba in Speech: Towards an Alternative to Self-Attention](https://mlsanity.com/paper/2405.12609/) (2024)
- [A Wavenet for Speech Denoising](https://mlsanity.com/paper/1706.07162/) (2017)
- [SELM: Speech Enhancement Using Discrete Tokens and Language Models](https://mlsanity.com/paper/2312.09747/) (2023)
- [A Hybrid DSP/Deep Learning Approach to Real-Time Full-Band Speech Enhancement](https://mlsanity.com/paper/1709.08243/) (2017)
- [Speech Denoising with Deep Feature Losses](https://mlsanity.com/paper/1806.10522/) (2018)
- [FRCRN: Boosting Feature Representation using Frequency Recurrence for Monaural Speech Enhancement](https://mlsanity.com/paper/2206.07293/) (2022)
- [DeepFilterNet: Perceptually Motivated Real-Time Speech Enhancement](https://mlsanity.com/paper/2305.08227/) (2023)
- [A Perceptually-Motivated Approach for Low-Complexity, Real-Time Enhancement of Fullband Speech](https://mlsanity.com/paper/2008.04259/) (2020)
- [Spiking Structured State Space Model for Monaural Speech Enhancement](https://mlsanity.com/paper/2309.03641/) (2023)
- [Speech Denoising Without Clean Training Data: A Noise2Noise Approach](https://mlsanity.com/paper/2104.03838/) (2021)
- [A Lightweight Spatiotemporal Network for Online Eye Tracking with Event Camera](https://mlsanity.com/paper/2404.08858/) (2024)
- [Speech Denoising in the Waveform Domain with Self-Attention](https://mlsanity.com/paper/2202.07790/) (2022)
- [A Multi-dimensional Deep Structured State Space Approach to Speech Enhancement Using Small-footprint Models](https://mlsanity.com/paper/2306.00331/) (2023)
- [A Review of Multi-Objective Deep Learning Speech Denoising Methods](https://mlsanity.com/paper/2003.12108/) (2020)
- [Convolutional Neural Networks to Enhance Coded Speech](https://mlsanity.com/paper/1806.09411/) (2018)
- [Speech Enhancement based on Denoising Autoencoder with Multi-branched Encoders](https://mlsanity.com/paper/2001.01538/) (2020)
- [PLEIADES: Building Temporal Kernels with Orthogonal Polynomials](https://mlsanity.com/paper/2405.12179/) (2024)

## Abstract (arXiv)

We present aTENNuate, a simple deep state-space autoencoder configured for efficient online raw speech enhancement in an end-to-end fashion. The network's performance is primarily evaluated on raw speech denoising, with additional assessments on tasks such as super-resolution and de-quantization. We benchmark aTENNuate on the VoiceBank + DEMAND and the Microsoft DNS1 synthetic test sets. The network outperforms previous real-time denoising models in terms of PESQ score, parameter count, MACs, and latency. Even as a raw waveform processing model, the model maintains high fidelity to the clean signal with minimal audible artifacts. In addition, the model remains performant even when the noisy input is compressed down to 4000Hz and 4 bits, suggesting general speech enhancement capabilities in low-resource environments. Try it out by pip install attenuate