# CMGAN: Conformer-based Metric GAN for Speech Enhancement

arXiv: https://arxiv.org/abs/2203.15149 · HTML: https://mlsanity.com/paper/2203.15149/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 40 references that are in the map.
- First posted on arXiv 28 Mar 2022.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Supervised Speech Separation Based on Deep Learning: An Overview](https://mlsanity.com/paper/1708.07524/) (2017)
- [Real Time Speech Enhancement in the Waveform Domain](https://mlsanity.com/paper/2006.12847/) (2020)
- [SEGAN: Speech Enhancement Generative Adversarial Network](https://mlsanity.com/paper/1703.09452/) (2017)
- [MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement](https://mlsanity.com/paper/2104.03538/) (2021)
- [MetricGAN: Generative Adversarial Networks based Black-box Metric Scores Optimization for Speech Enhancement](https://mlsanity.com/paper/1905.04874/) (2019)
- [Dual-Path Transformer Network: Direct Context-Aware Modeling for End-to-End Monaural Speech Separation](https://mlsanity.com/paper/2007.13975/) (2020)
- [ICASSP 2022 Deep Noise Suppression Challenge](https://mlsanity.com/paper/2202.13288/) (2022)
- [PHASEN: A Phase-and-Harmonics-Aware Speech Enhancement Network](https://mlsanity.com/paper/1911.04697/) (2019)
- [Continuous Speech Separation with Conformer](https://mlsanity.com/paper/2008.05773/) (2020)
- [TSTNN: Two-stage Transformer based Neural Network for Speech Enhancement in the Time Domain](https://mlsanity.com/paper/2103.09963/) (2021)
- [DPT-FSNet: Dual-path Transformer Based Full-band and Sub-band Fusion Network for Speech Enhancement](https://mlsanity.com/paper/2104.13002/) (2021)
- [Glance and Gaze: A Collaborative Learning Framework for Single-channel Speech Enhancement](https://mlsanity.com/paper/2106.11789/) (2021)
- [Improved Speech Enhancement with the Wave-U-Net](https://mlsanity.com/paper/1811.11307/) (2018)
- [A consolidated view of loss functions for supervised deep learning-based speech enhancement](https://mlsanity.com/paper/2009.12286/) (2020)
- [Dual-branch Attention-In-Attention Transformer for single-channel speech enhancement](https://mlsanity.com/paper/2110.06467/) (2021)
- [On The Compensation Between Magnitude and Phase in Speech Separation](https://mlsanity.com/paper/2108.05470/) (2021)
- [Exploring Tradeoffs in Models for Low-latency Speech Enhancement](https://mlsanity.com/paper/1811.07030/) (2018)
- [Investigating Cross-Domain Losses for Speech Enhancement](https://mlsanity.com/paper/2010.10468/) (2020)

## Built on by (most cited first, 30 of 45)

- [Speech Enhancement and Dereverberation with Diffusion-based Generative Models](https://mlsanity.com/paper/2208.05830/) (2022)
- [LauraGPT: Listen, Attend, Understand, and Regenerate Audio with GPT](https://mlsanity.com/paper/2310.04673/) (2023)
- [Separate Anything You Describe](https://mlsanity.com/paper/2308.05037/) (2023)
- [CMGAN: Conformer-Based Metric-GAN for Monaural Speech Enhancement](https://mlsanity.com/paper/2209.11112/) (2022)
- [The PESQetarian: On the Relevance of Goodhart's Law for Speech Enhancement](https://mlsanity.com/paper/2406.03460/) (2024)
- [Mamba-SEUNet: Mamba UNet for Monaural Speech Enhancement](https://mlsanity.com/paper/2412.16626/) (2024)
- [SCP-GAN: Self-Correcting Discriminator Optimization for Training Consistency Preserving Metric GAN on Speech Enhancement Tasks](https://mlsanity.com/paper/2210.14474/) (2022)
- [Noise robust speech emotion recognition with signal-to-noise ratio adapting speech enhancement](https://mlsanity.com/paper/2309.01164/) (2023)
- [A Variance-Preserving Interpolation Approach for Diffusion Models with Applications to Single Channel Speech Enhancement and Recognition](https://mlsanity.com/paper/2405.16952/) (2024)
- [MUSE: Flexible Voiceprint Receptive Fields and Multi-Path Fusion Enhanced Taylor Transformer for U-Net-based Speech Enhancement](https://mlsanity.com/paper/2406.04589/) (2024)
- [Deep neural network techniques for monaural speech enhancement: state of the art analysis](https://mlsanity.com/paper/2212.00369/) (2022)
- [Multi-Dimensional and Multi-Scale Modeling for Speech Separation Optimized by Discriminative Learning](https://mlsanity.com/paper/2303.03737/) (2023)
- [Large-scale unsupervised audio pre-training for video-to-speech synthesis](https://mlsanity.com/paper/2306.15464/) (2023)
- [A Survey of Deep Learning for Complex Speech Spectrograms](https://mlsanity.com/paper/2505.08694/) (2025)
- [DPATD: Dual-Phase Audio Transformer for Denoising](https://mlsanity.com/paper/2310.19588/) (2023)
- [From KAN to GR-KAN: Advancing Speech Enhancement with KAN-Based Methodology](https://mlsanity.com/paper/2412.17778/) (2024)
- [A Two-Stage Hierarchical Deep Filtering Framework for Real-Time Speech Enhancement](https://mlsanity.com/paper/2506.01023/) (2025)
- [WhiSQA: Non-Intrusive Speech Quality Prediction Using Whisper Encoder Features](https://mlsanity.com/paper/2508.02210/) (2025)
- [A Training and Inference Strategy Using Noisy and Enhanced Speech as Target for Speech Enhancement without Clean Speech](https://mlsanity.com/paper/2210.15368/) (2022)
- [Audio Denoising for Robust Audio Fingerprinting](https://mlsanity.com/paper/2212.11277/) (2022)
- [Diffusion Gaussian Mixture Audio Denoise](https://mlsanity.com/paper/2406.09154/) (2024)
- [Complex Image-Generative Diffusion Transformer for Audio Denoising](https://mlsanity.com/paper/2406.09161/) (2024)
- [Effective Noise-aware Data Simulation for Domain-adaptive Speech Enhancement Leveraging Dynamic Stochastic Perturbation](https://mlsanity.com/paper/2409.01545/) (2024)
- [Collective Learning Mechanism based Optimal Transport Generative Adversarial Network for Non-parallel Voice Conversion](https://mlsanity.com/paper/2504.13791/) (2025)
- [DeepFilterGAN: A Full-band Real-time Speech Enhancement System with GAN-based Stochastic Regeneration](https://mlsanity.com/paper/2505.23515/) (2025)
- [Improved Intelligibility of Dysarthric Speech using Conditional Flow Matching](https://mlsanity.com/paper/2506.16127/) (2025)
- [Leveraging Discriminative Latent Representations for Conditioning GAN-Based Speech Enhancement](https://mlsanity.com/paper/2508.20859/) (2025)
- [Quaternion Self-Attention with Shared Scores](https://mlsanity.com/paper/2605.24920/) (2026)
- [Inference and Denoise: Causal Inference-based Neural Speech Enhancement](https://mlsanity.com/paper/2211.01189/) (2022)
- [Music Augmentation and Denoising For Peak-Based Audio Fingerprinting](https://mlsanity.com/paper/2310.13388/) (2023)

## Abstract (arXiv)

Recently, convolution-augmented transformer (Conformer) has achieved promising performance in automatic speech recognition (ASR) and time-domain speech enhancement (SE), as it can capture both local and global dependencies in the speech signal. In this paper, we propose a conformer-based metric generative adversarial network (CMGAN) for SE in the time-frequency (TF) domain. In the generator, we utilize two-stage conformer blocks to aggregate all magnitude and complex spectrogram information by modeling both time and frequency dependencies. The estimation of magnitude and complex spectrogram is decoupled in the decoder stage and then jointly incorporated to reconstruct the enhanced speech. In addition, a metric discriminator is employed to further improve the quality of the enhanced estimated speech by optimizing the generator with respect to a corresponding evaluation score. Quantitative analysis on Voice Bank+DEMAND dataset indicates the capability of CMGAN in outperforming various previous models with a margin, i.e., PESQ of 3.41 and SSNR of 11.10 dB.