# Speech Denoising in the Waveform Domain with Self-Attention

arXiv: https://arxiv.org/abs/2202.07790 · HTML: https://mlsanity.com/paper/2202.07790/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 41 references that are in the map.
- First posted on arXiv 15 Feb 2022.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation](https://mlsanity.com/paper/1809.07454/) (2018)
- [Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram](https://mlsanity.com/paper/1910.11480/) (2019)
- [Real Time Speech Enhancement in the Waveform Domain](https://mlsanity.com/paper/2006.12847/) (2020)
- [SEGAN: Speech Enhancement Generative Adversarial Network](https://mlsanity.com/paper/1703.09452/) (2017)
- [The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Testing Framework, and Challenge Results](https://mlsanity.com/paper/2005.13981/) (2020)
- [Wave-U-Net: A Multi-Scale Neural Network for End-to-End Audio Source Separation](https://mlsanity.com/paper/1806.03185/) (2018)
- [MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement](https://mlsanity.com/paper/2104.03538/) (2021)
- [MetricGAN: Generative Adversarial Networks based Black-box Metric Scores Optimization for Speech Enhancement](https://mlsanity.com/paper/1905.04874/) (2019)
- [U-Net Transformer: Self and Cross Attention for Medical Image Segmentation](https://mlsanity.com/paper/2103.06104/) (2021)
- [ClariNet: Parallel Wave Generation in End-to-End Text-to-Speech](https://mlsanity.com/paper/1807.07281/) (2018)
- [FullSubNet: A Full-Band and Sub-Band Fusion Model for Real-Time Single-Channel Speech Enhancement](https://mlsanity.com/paper/2010.15508/) (2020)
- [A Wavenet for Speech Denoising](https://mlsanity.com/paper/1706.07162/) (2017)
- [Speech Denoising with Deep Feature Losses](https://mlsanity.com/paper/1806.10522/) (2018)
- [PoCoNet: Better Speech Enhancement with Frequency-Positional Embeddings, Semi-Supervised Conversational Data, and Biased Loss](https://mlsanity.com/paper/2008.04470/) (2020)
- [Improving GANs for Speech Enhancement](https://mlsanity.com/paper/2001.05532/) (2020)
- [Dual-Signal Transformation LSTM Network for Real-Time Noise Suppression](https://mlsanity.com/paper/2005.07551/) (2020)
- [UNetGAN: A Robust Speech Enhancement Approach in Time Domain for Extremely Low Signal-to-noise Ratio Condition](https://mlsanity.com/paper/2010.15521/) (2020)
- [Multi-Objective Learning and Mask-Based Post-Processing for Deep Neural Network Based Speech Enhancement](https://mlsanity.com/paper/1703.07172/) (2017)

## Built on by (most cited first, 13 of 13)

- [Mamba in Speech: Towards an Alternative to Self-Attention](https://mlsanity.com/paper/2405.12609/) (2024)
- [TUNeS: A Temporal U-Net with Self-Attention for Video-based Surgical Phase Recognition](https://mlsanity.com/paper/2307.09997/) (2023)
- [RF Challenge: The Data-Driven Radio Frequency Signal Separation Challenge](https://mlsanity.com/paper/2409.08839/) (2024)
- [SAV-SE: Scene-aware Audio-Visual Speech Enhancement with Selective State Space Model](https://mlsanity.com/paper/2411.07751/) (2024)
- [JaCappella Corpus: A Japanese a Cappella Vocal Ensemble Corpus](https://mlsanity.com/paper/2211.16028/) (2022)
- [Deep neural network techniques for monaural speech enhancement: state of the art analysis](https://mlsanity.com/paper/2212.00369/) (2022)
- [Complex Image Generation SwinTransformer Network for Audio Denoising](https://mlsanity.com/paper/2310.16109/) (2023)
- [CleanUNet 2: A Hybrid Speech Denoising Model on Waveform and Spectrogram](https://mlsanity.com/paper/2309.05975/) (2023)
- [CleanUMamba: A Compact Mamba Network for Speech Denoising using Channel Pruning](https://mlsanity.com/paper/2410.11062/) (2024)
- [VCoME: Verbal Video Composition with Multimodal Editing Effects](https://mlsanity.com/paper/2407.04697/) (2024)
- [Selective State Space Model for Monaural Speech Enhancement](https://mlsanity.com/paper/2411.06217/) (2024)
- [Efficient Audiovisual Speech Processing via MUTUD: Multimodal Training and Unimodal Deployment](https://mlsanity.com/paper/2501.18157/) (2025)
- [aTENNuate: Optimized Real-time Speech Enhancement with Deep SSMs on Raw Audio](https://mlsanity.com/paper/2409.03377/) (2024)

## Abstract (arXiv)

In this work, we present CleanUNet, a causal speech denoising model on the raw waveform. The proposed model is based on an encoder-decoder architecture combined with several self-attention blocks to refine its bottleneck representations, which is crucial to obtain good results. The model is optimized through a set of losses defined over both waveform and multi-resolution spectrograms. The proposed method outperforms the state-of-the-art models in terms of denoised speech quality from various objective and subjective evaluation metrics. We release our code and models at https://github.com/nvidia/cleanunet.