# Efficient Monaural Speech Enhancement using Spectrum Attention Fusion

arXiv: https://arxiv.org/abs/2308.02263 · HTML: https://mlsanity.com/paper/2308.02263/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 41 references that are in the map.
- First posted on arXiv 4 Aug 2023.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Supervised Speech Separation Based on Deep Learning: An Overview](https://mlsanity.com/paper/1708.07524/) (2017)
- [DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancement](https://mlsanity.com/paper/2008.00264/) (2020)
- [Deep Learning for Audio Signal Processing](https://mlsanity.com/paper/1905.00078/) (2019)
- [Conv2Former: A Simple Transformer-Style ConvNet for Visual Recognition](https://mlsanity.com/paper/2211.11943/) (2022)
- [Phase-aware Speech Enhancement with Deep Complex U-Net](https://mlsanity.com/paper/1903.03107/) (2019)
- [FullSubNet: A Full-Band and Sub-Band Fusion Model for Real-Time Single-Channel Speech Enhancement](https://mlsanity.com/paper/2010.15508/) (2020)
- [A Wavenet for Speech Denoising](https://mlsanity.com/paper/1706.07162/) (2017)
- [DeepFilterNet: A Low Complexity Speech Enhancement Framework for Full-Band Audio based on Deep Filtering](https://mlsanity.com/paper/2110.05588/) (2021)
- [A Hybrid DSP/Deep Learning Approach to Real-Time Full-Band Speech Enhancement](https://mlsanity.com/paper/1709.08243/) (2017)
- [Towards efficient models for real-time deep noise suppression](https://mlsanity.com/paper/2101.09249/) (2021)
- [FullSubNet+: Channel Attention FullSubNet with Complex Spectrograms for Speech Enhancement](https://mlsanity.com/paper/2203.12188/) (2022)
- [Glance and Gaze: A Collaborative Learning Framework for Single-channel Speech Enhancement](https://mlsanity.com/paper/2106.11789/) (2021)
- [A Fully Convolutional Neural Network for Speech Enhancement](https://mlsanity.com/paper/1609.07132/) (2016)
- [A Perceptually-Motivated Approach for Low-Complexity, Real-Time Enhancement of Fullband Speech](https://mlsanity.com/paper/2008.04259/) (2020)
- [DCCRN+: Channel-wise Subband DCCRN with SNR Estimation for Speech Enhancement](https://mlsanity.com/paper/2106.08672/) (2021)
- [S-DCCRN: Super Wide Band DCCRN with learnable complex feature for speech enhancement](https://mlsanity.com/paper/2111.08387/) (2021)

## Abstract (arXiv)

Speech enhancement is a demanding task in automated speech processing pipelines, focusing on separating clean speech from noisy channels. Transformer based models have recently bested RNN and CNN models in speech enhancement, however at the same time they are much more computationally expensive and require much more high quality training data, which is always hard to come by. In this paper, we present an improvement for speech enhancement models that maintains the expressiveness of self-attention while significantly reducing model complexity, which we have termed Spectrum Attention Fusion. We carefully construct a convolutional module to replace several self-attention layers in a speech Transformer, allowing the model to more efficiently fuse spectral features. Our proposed model is able to achieve comparable or better results against SOTA models but with significantly smaller parameters (0.58M) on the Voice Bank + DEMAND dataset.