# CleanUNet 2: A Hybrid Speech Denoising Model on Waveform and Spectrogram

arXiv: https://arxiv.org/abs/2309.05975 · HTML: https://mlsanity.com/paper/2309.05975/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 46 references that are in the map.
- First posted on arXiv 12 Sep 2023.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions](https://mlsanity.com/paper/1712.05884/) (2017)
- [Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation](https://mlsanity.com/paper/1809.07454/) (2018)
- [Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram](https://mlsanity.com/paper/1910.11480/) (2019)
- [Real Time Speech Enhancement in the Waveform Domain](https://mlsanity.com/paper/2006.12847/) (2020)
- [SEGAN: Speech Enhancement Generative Adversarial Network](https://mlsanity.com/paper/1703.09452/) (2017)
- [The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Testing Framework, and Challenge Results](https://mlsanity.com/paper/2005.13981/) (2020)
- [Wave-U-Net: A Multi-Scale Neural Network for End-to-End Audio Source Separation](https://mlsanity.com/paper/1806.03185/) (2018)
- [MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement](https://mlsanity.com/paper/2104.03538/) (2021)
- [MetricGAN: Generative Adversarial Networks based Black-box Metric Scores Optimization for Speech Enhancement](https://mlsanity.com/paper/1905.04874/) (2019)
- [ClariNet: Parallel Wave Generation in End-to-End Text-to-Speech](https://mlsanity.com/paper/1807.07281/) (2018)
- [FullSubNet: A Full-Band and Sub-Band Fusion Model for Real-Time Single-Channel Speech Enhancement](https://mlsanity.com/paper/2010.15508/) (2020)
- [WaveFlow: A Compact Flow-based Model for Raw Audio](https://mlsanity.com/paper/1912.01219/) (2019)
- [A Wavenet for Speech Denoising](https://mlsanity.com/paper/1706.07162/) (2017)
- [Speech Denoising with Deep Feature Losses](https://mlsanity.com/paper/1806.10522/) (2018)
- [Glance and Gaze: A Collaborative Learning Framework for Single-channel Speech Enhancement](https://mlsanity.com/paper/2106.11789/) (2021)
- [Raw Waveform-based Speech Enhancement by Fully Convolutional Networks](https://mlsanity.com/paper/1703.02205/) (2017)
- [PoCoNet: Better Speech Enhancement with Frequency-Positional Embeddings, Semi-Supervised Conversational Data, and Biased Loss](https://mlsanity.com/paper/2008.04470/) (2020)
- [Dual-Signal Transformation LSTM Network for Real-Time Noise Suppression](https://mlsanity.com/paper/2005.07551/) (2020)
- [Speech Denoising in the Waveform Domain with Self-Attention](https://mlsanity.com/paper/2202.07790/) (2022)
- [UNetGAN: A Robust Speech Enhancement Approach in Time Domain for Extremely Low Signal-to-noise Ratio Condition](https://mlsanity.com/paper/2010.15521/) (2020)
- [Multi-Objective Learning and Mask-Based Post-Processing for Deep Neural Network Based Speech Enhancement](https://mlsanity.com/paper/1703.07172/) (2017)

## Built on by (most cited first, 2 of 2)

- [Test-Time Training for Speech Enhancement](https://mlsanity.com/paper/2508.01847/) (2025)
- [Universal Robust Speech Adaptation for Cross-Domain Speech Recognition and Enhancement](https://mlsanity.com/paper/2602.04307/) (2026)

## Abstract (arXiv)

In this work, we present CleanUNet 2, a speech denoising model that combines the advantages of waveform denoiser and spectrogram denoiser and achieves the best of both worlds. CleanUNet 2 uses a two-stage framework inspired by popular speech synthesis methods that consist of a waveform model and a spectrogram model. Specifically, CleanUNet 2 builds upon CleanUNet, the state-of-the-art waveform denoiser, and further boosts its performance by taking predicted spectrograms from a spectrogram denoiser as the input. We demonstrate that CleanUNet 2 outperforms previous methods in terms of various objective and subjective evaluations.