# Phase-aware Speech Enhancement with Deep Complex U-Net

arXiv: https://arxiv.org/abs/1903.03107 · HTML: https://mlsanity.com/paper/1903.03107/

## Key facts

- Cited by 73 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Mar 2019.

## Built on by (most cited first, 30 of 73)

- [Speech Enhancement and Dereverberation with Diffusion-based Generative Models](https://mlsanity.com/paper/2208.05830/) (2022)
- [Speech Enhancement with Score-Based Generative Models in the Complex STFT Domain](https://mlsanity.com/paper/2203.17004/) (2022)
- [Audio Self-supervised Learning: A Survey](https://mlsanity.com/paper/2203.01205/) (2022)
- [Visual Acoustic Matching](https://mlsanity.com/paper/2202.06875/) (2022)
- [TSTNN: Two-stage Transformer based Neural Network for Speech Enhancement in the Time Domain](https://mlsanity.com/paper/2103.09963/) (2021)
- [Looking into Your Speech: Learning Cross-modal Affinity for Audio-visual Speech Separation](https://mlsanity.com/paper/2104.02775/) (2021)
- [Personalized Speech Enhancement: New Models and Comprehensive Evaluation](https://mlsanity.com/paper/2110.09625/) (2021)
- [RemixIT: Continual self-training of speech enhancement models via bootstrapped remixing](https://mlsanity.com/paper/2202.08862/) (2022)
- [Dual-branch Attention-In-Attention Transformer for single-channel speech enhancement](https://mlsanity.com/paper/2110.06467/) (2021)
- [Speech Denoising Without Clean Training Data: A Noise2Noise Approach](https://mlsanity.com/paper/2104.03838/) (2021)
- [Real-time Denoising and Dereverberation with Tiny Recurrent U-Net](https://mlsanity.com/paper/2102.03207/) (2021)
- [Building Blocks for a Complex-Valued Transformer Architecture](https://mlsanity.com/paper/2306.09827/) (2023)
- [Diffiner: A Versatile Diffusion-based Generative Refiner for Speech Enhancement](https://mlsanity.com/paper/2210.17287/) (2022)
- [TRAMBA: A Hybrid Transformer and Mamba Architecture for Practical Audio and Bone Conduction Speech Super Resolution and Enhancement on Mobile and Wearable Platforms](https://mlsanity.com/paper/2405.01242/) (2024)
- [A Multi-dimensional Deep Structured State Space Approach to Speech Enhancement Using Small-footprint Models](https://mlsanity.com/paper/2306.00331/) (2023)
- [Reverb Conversion of Mixed Vocal Tracks Using an End-to-end Convolutional Deep Neural Network](https://mlsanity.com/paper/2103.02147/) (2021)
- [FFC-SE: Fast Fourier Convolution for Speech Enhancement](https://mlsanity.com/paper/2204.03042/) (2022)
- [Monaural Speech Enhancement with Complex Convolutional Block Attention Module and Joint Time Frequency Losses](https://mlsanity.com/paper/2102.01993/) (2021)
- [Rethinking complex-valued deep neural networks for monaural speech enhancement](https://mlsanity.com/paper/2301.04320/) (2023)
- [Complex Mixer for MedMNIST Classification Decathlon](https://mlsanity.com/paper/2304.10054/) (2023)
- [Assessing the Generalization Gap of Learning-Based Speech Enhancement Systems in Noisy and Reverberant Environments](https://mlsanity.com/paper/2309.06183/) (2023)
- [SonicMaster: Towards Controllable All-in-One Music Restoration and Mastering](https://mlsanity.com/paper/2508.03448/) (2025)
- [Efficient Personalized Speech Enhancement through Self-Supervised Learning](https://mlsanity.com/paper/2104.02017/) (2021)
- [Variance-Preserving-Based Interpolation Diffusion Models for Speech Enhancement](https://mlsanity.com/paper/2306.08527/) (2023)
- [DPSNN: Spiking Neural Network for Low-Latency Streaming Speech Enhancement](https://mlsanity.com/paper/2408.07388/) (2024)
- [BCNN: Binary Complex Neural Network](https://mlsanity.com/paper/2104.10044/) (2021)
- [Adversarial Audio Synthesis with Complex-valued Polynomial Networks](https://mlsanity.com/paper/2206.06811/) (2022)
- [Diffusion-based Signal Refiner for Speech Enhancement and Separation](https://mlsanity.com/paper/2305.05857/) (2023)
- [A Mask Free Neural Network for Monaural Speech Enhancement](https://mlsanity.com/paper/2306.04286/) (2023)
- [Complex Image Generation SwinTransformer Network for Audio Denoising](https://mlsanity.com/paper/2310.16109/) (2023)

## Abstract (arXiv)

Most deep learning-based models for speech enhancement have mainly focused on estimating the magnitude of spectrogram while reusing the phase from noisy speech for reconstruction. This is due to the difficulty of estimating the phase of clean speech. To improve speech enhancement performance, we tackle the phase estimation problem in three ways. First, we propose Deep Complex U-Net, an advanced U-Net structured model incorporating well-defined complex-valued building blocks to deal with complex-valued spectrograms. Second, we propose a polar coordinate-wise complex-valued masking method to reflect the distribution of complex ideal ratio masks. Third, we define a novel loss function, weighted source-to-distortion ratio (wSDR) loss, which is designed to directly correlate with a quantitative evaluation measure. Our model was evaluated on a mixture of the Voice Bank corpus and DEMAND database, which has been widely used by many deep learning models for speech enhancement. Ablation experiments were conducted on the mixed dataset showing that all three proposed approaches are empirically valid. Experimental results show that the proposed method achieves state-of-the-art performance in all metrics, outperforming previous approaches by a large margin.