# MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement

arXiv: https://arxiv.org/abs/2104.03538 · HTML: https://mlsanity.com/paper/2104.03538/

## Key facts

- Cited by 114 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 44 references that are in the map.
- First posted on arXiv 8 Apr 2021.

## Builds on (most cited first, 26 of 26)

- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressors](https://mlsanity.com/paper/2010.15258/) (2020)
- [Supervised Speech Separation Based on Deep Learning: An Overview](https://mlsanity.com/paper/1708.07524/) (2017)
- [Real Time Speech Enhancement in the Waveform Domain](https://mlsanity.com/paper/2006.12847/) (2020)
- [SEGAN: Speech Enhancement Generative Adversarial Network](https://mlsanity.com/paper/1703.09452/) (2017)
- [HashNet: Deep Learning to Hash by Continuation](https://mlsanity.com/paper/1702.00758/) (2017)
- [MetricGAN: Generative Adversarial Networks based Black-box Metric Scores Optimization for Speech Enhancement](https://mlsanity.com/paper/1905.04874/) (2019)
- [PHASEN: A Phase-and-Harmonics-Aware Speech Enhancement Network](https://mlsanity.com/paper/1911.04697/) (2019)
- [HiFi-GAN: High-Fidelity Denoising and Dereverberation Based on Speech Deep Features in Adversarial Networks](https://mlsanity.com/paper/2006.05694/) (2020)
- [Quality-Net: An End-to-End Non-intrusive Speech Quality Assessment Model based on BLSTM](https://mlsanity.com/paper/1808.05344/) (2018)
- [Speech Enhancement using Self-Adaptation and Multi-Head Self-Attention](https://mlsanity.com/paper/2002.05873/) (2020)
- [End-to-End Waveform Utterance Enhancement for Direct Evaluation Metrics Optimization by Fully Convolutional Neural Networks](https://mlsanity.com/paper/1709.03658/) (2017)
- [A Differentiable Perceptual Audio Metric Learned from Just Noticeable Differences](https://mlsanity.com/paper/2001.04460/) (2020)
- [Perceptual Loss based Speech Denoising with an ensemble of Audio Pattern Recognition and Self-Supervised Models](https://mlsanity.com/paper/2010.11860/) (2020)
- [Spectral feature mapping with mimic loss for robust speech recognition](https://mlsanity.com/paper/1803.09816/) (2018)
- [DNN-based Source Enhancement to Increase Objective Sound Quality Assessment Score](https://mlsanity.com/paper/1810.09137/) (2018)
- [End-to-End Multi-Task Denoising for joint SDR and PESQ Optimization](https://mlsanity.com/paper/1901.09146/) (2019)
- [Learning with Learned Loss Function: Speech Enhancement with Quality-Net to Improve Perceptual Evaluation of Speech Quality](https://mlsanity.com/paper/1905.01898/) (2019)
- [SERIL: Noise Adaptive Speech Enhancement using Regularization-based Incremental Learning](https://mlsanity.com/paper/2005.11760/) (2020)
- [iMetricGAN: Intelligibility Enhancement for Speech-in-Noise using Generative Adversarial Network-based Metric Learning](https://mlsanity.com/paper/2004.00932/) (2020)
- [Monaural Speech Enhancement using Deep Neural Networks by Maximizing a Short-Time Objective Intelligibility Measure](https://mlsanity.com/paper/1802.00604/) (2018)
- [Boosting Objective Scores of a Speech Enhancement Model by MetricGAN Post-processing](https://mlsanity.com/paper/2006.10296/) (2020)
- [Acoustics-guided evaluation (AGE): a new measure for estimating performance of speech enhancement algorithms for robust ASR](https://mlsanity.com/paper/1811.11517/) (2018)
- [Stable Training of DNN for Speech Enhancement based on Perceptually-Motivated Black-Box Cost Function](https://mlsanity.com/paper/2002.05879/) (2020)

## Built on by (most cited first, 30 of 114)

- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [Speech Enhancement and Dereverberation with Diffusion-based Generative Models](https://mlsanity.com/paper/2208.05830/) (2022)
- [StoRM: A Diffusion-based Stochastic Regeneration Model for Speech Enhancement and Dereverberation](https://mlsanity.com/paper/2212.11851/) (2022)
- [Universal Speech Enhancement with Score-based Diffusion](https://mlsanity.com/paper/2206.03065/) (2022)
- [An Investigation of Incorporating Mamba for Speech Enhancement](https://mlsanity.com/paper/2405.06573/) (2024)
- [Mamba in Speech: Towards an Alternative to Self-Attention](https://mlsanity.com/paper/2405.12609/) (2024)
- [Generative Pre-training for Speech with Flow Matching](https://mlsanity.com/paper/2310.16338/) (2023)
- [HiFi++: a Unified Framework for Bandwidth Extension and Speech Enhancement](https://mlsanity.com/paper/2203.13086/) (2022)
- [CMGAN: Conformer-based Metric GAN for Speech Enhancement](https://mlsanity.com/paper/2203.15149/) (2022)
- [Unsupervised Speech Enhancement using Dynamical Variational Auto-Encoders](https://mlsanity.com/paper/2106.12271/) (2021)
- [Learning Audio-Visual Dereverberation](https://mlsanity.com/paper/2106.07732/) (2021)
- [CMGAN: Conformer-Based Metric-GAN for Monaural Speech Enhancement](https://mlsanity.com/paper/2209.11112/) (2022)
- [Spiking Structured State Space Model for Monaural Speech Enhancement](https://mlsanity.com/paper/2309.03641/) (2023)
- [AV-RIR: Audio-Visual Room Impulse Response Estimation](https://mlsanity.com/paper/2312.00834/) (2023)
- [MetricGAN-U: Unsupervised speech enhancement/ dereverberation based only on noisy/ reverberated speech](https://mlsanity.com/paper/2110.05866/) (2021)
- [Dual-branch Attention-In-Attention Transformer for single-channel speech enhancement](https://mlsanity.com/paper/2110.06467/) (2021)
- [Unsupervised Noise Adaptive Speech Enhancement by Discriminator-Constrained Optimal Transport](https://mlsanity.com/paper/2111.06316/) (2021)
- [AdVerb: Visually Guided Audio Dereverberation](https://mlsanity.com/paper/2308.12370/) (2023)
- [Self-Supervised Visual Acoustic Matching](https://mlsanity.com/paper/2307.15064/) (2023)
- [Speech Denoising in the Waveform Domain with Self-Attention](https://mlsanity.com/paper/2202.07790/) (2022)
- [The PESQetarian: On the Relevance of Goodhart's Law for Speech Enhancement](https://mlsanity.com/paper/2406.03460/) (2024)
- [Discretization and Re-synthesis: an alternative method to solve the Cocktail Party Problem](https://mlsanity.com/paper/2112.09382/) (2021)
- [Diffusion-Based Speech Enhancement with Joint Generative and Predictive Decoders](https://mlsanity.com/paper/2305.10734/) (2023)
- [BSS-CFFMA: Cross-Domain Feature Fusion and Multi-Attention Speech Enhancement Network based on Self-Supervised Embedding](https://mlsanity.com/paper/2408.06851/) (2024)
- [Mamba-SEUNet: Mamba UNet for Monaural Speech Enhancement](https://mlsanity.com/paper/2412.16626/) (2024)
- [A Multi-dimensional Deep Structured State Space Approach to Speech Enhancement Using Small-footprint Models](https://mlsanity.com/paper/2306.00331/) (2023)
- [Noise-aware Speech Enhancement using Diffusion Probabilistic Model](https://mlsanity.com/paper/2307.08029/) (2023)
- [Wave-U-Mamba: An End-To-End Framework For High-Quality And Efficient Speech Super Resolution](https://mlsanity.com/paper/2409.09337/) (2024)
- [Music Separation Enhancement with Generative Modeling](https://mlsanity.com/paper/2208.12387/) (2022)
- [SCP-GAN: Self-Correcting Discriminator Optimization for Training Consistency Preserving Metric GAN on Speech Enhancement Tasks](https://mlsanity.com/paper/2210.14474/) (2022)

## Abstract (arXiv)

The discrepancy between the cost function used for training a speech enhancement model and human auditory perception usually makes the quality of enhanced speech unsatisfactory. Objective evaluation metrics which consider human perception can hence serve as a bridge to reduce the gap. Our previously proposed MetricGAN was designed to optimize objective metrics by connecting the metric with a discriminator. Because only the scores of the target evaluation functions are needed during training, the metrics can even be non-differentiable. In this study, we propose a MetricGAN+ in which three training techniques incorporating domain-knowledge of speech processing are proposed. With these techniques, experimental results on the VoiceBank-DEMAND dataset show that MetricGAN+ can increase PESQ score by 0.3 compared to the previous MetricGAN and achieve state-of-the-art results (PESQ score = 3.15).