# Joint magnitude estimation and phase recovery using Cycle-in-Cycle GAN for non-parallel speech enhancement

arXiv: https://arxiv.org/abs/2109.12591 · HTML: https://mlsanity.com/paper/2109.12591/

## Key facts

- Cited by 2 papers in the map (data through 19 Oct 2026).
- Placed in Speech enhancement and noise suppression, one of 11 directions in Speech and audio language models.
- Builds on 16 of its 40 references that are in the map.
- First posted on arXiv 26 Sep 2021.

## Placed in

- Speech and audio language models / [Speech enhancement and noise suppression](https://mlsanity.com/speech/speech-enhancement-and-noise-suppression/)

## Builds on (most cited first, 16 of 16)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressors](https://mlsanity.com/paper/2010.15258/) (2020)
- [Supervised Speech Separation Based on Deep Learning: An Overview](https://mlsanity.com/paper/1708.07524/) (2017)
- [DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancement](https://mlsanity.com/paper/2008.00264/) (2020)
- [SEGAN: Speech Enhancement Generative Adversarial Network](https://mlsanity.com/paper/1703.09452/) (2017)
- [The relativistic discriminator: a key element missing from standard GAN](https://mlsanity.com/paper/1807.00734/) (2018)
- [Deep Complex Networks](https://mlsanity.com/paper/1705.09792/) (2017)
- [MetricGAN: Generative Adversarial Networks based Black-box Metric Scores Optimization for Speech Enhancement](https://mlsanity.com/paper/1905.04874/) (2019)
- [Wave-U-Net: A Multi-Scale Neural Network for End-to-End Audio Source Separation](https://mlsanity.com/paper/1806.03185/) (2018)
- [Speech Denoising with Deep Feature Losses](https://mlsanity.com/paper/1806.10522/) (2018)
- [A Simultaneous Denoising and Dereverberation Framework with Target Decoupling](https://mlsanity.com/paper/2106.12743/) (2021)
- [CycleGAN-based Non-parallel Speech Enhancement with an Adaptive Attention-in-attention Mechanism](https://mlsanity.com/paper/2107.13143/) (2021)
- [Cycle-Consistent Speech Enhancement](https://mlsanity.com/paper/1809.02253/) (2018)
- [On Loss Functions and Recurrency Training for GAN-based Speech Enhancement Systems](https://mlsanity.com/paper/2007.14974/) (2020)
- [A Two-stage Complex Network using Cycle-consistent Generative Adversarial Networks for Speech Enhancement](https://mlsanity.com/paper/2109.02011/) (2021)
- [Self-Attention Generative Adversarial Network for Speech Enhancement](https://mlsanity.com/paper/2010.09132/) (2020)

## Built on by (most cited first, 2 of 2)

- [DBT-Net: Dual-branch federative magnitude and phase estimation with attention-in-attention transformer for monaural speech enhancement](https://mlsanity.com/paper/2202.07931/) (2022)
- [Unsupervised Speech Enhancement via Drifting](https://mlsanity.com/paper/2609.34662/) (2026)

## Abstract (arXiv)

For the lack of adequate paired noisy-clean speech corpus in many real scenarios, non-parallel training is a promising task for DNN-based speech enhancement methods. However, because of the severe mismatch between input and target speeches, many previous studies only focus on the magnitude spectrum estimation and remain the phase unaltered, resulting in the degraded speech quality under low signal-to-noise ratio conditions. To tackle this problem, we decouple the difficult target w.r.t. original spectrum optimization into spectral magnitude and phase, and a novel Cycle-in-Cycle generative adversarial network (dubbed CinCGAN) is proposed to jointly estimate the spectral magnitude and phase information stage by stage under unpaired data. In the first stage, we pretrain a magnitude CycleGAN to coarsely estimate the spectral magnitude of clean speech. In the second stage, we incorporate the pretrained CycleGAN with a complex-valued CycleGAN as a cycle-in-cycle structure to simultaneously recover phase information and refine the overall spectrum. Experimental results demonstrate that the proposed approach significantly outperforms previous baselines under non-parallel training. The evaluation on training the models with standard paired data also shows that CinCGAN achieves remarkable performance especially in reducing background noise and speech distortion.