# The PESQetarian: On the Relevance of Goodhart's Law for Speech Enhancement

arXiv: https://arxiv.org/abs/2406.03460 · HTML: https://mlsanity.com/paper/2406.03460/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 32 references that are in the map.
- First posted on arXiv 5 Jun 2024.

## Builds on (most cited first, 17 of 17)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation](https://mlsanity.com/paper/1809.07454/) (2018)
- [Speech Enhancement and Dereverberation with Diffusion-based Generative Models](https://mlsanity.com/paper/2208.05830/) (2022)
- [DNSMOS P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressors](https://mlsanity.com/paper/2110.01763/) (2021)
- [MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement](https://mlsanity.com/paper/2104.03538/) (2021)
- [MetricGAN: Generative Adversarial Networks based Black-box Metric Scores Optimization for Speech Enhancement](https://mlsanity.com/paper/1905.04874/) (2019)
- [QuartzNet: Deep Automatic Speech Recognition with 1D Time-Channel Separable Convolutions](https://mlsanity.com/paper/1910.10261/) (2019)
- [CMGAN: Conformer-based Metric GAN for Speech Enhancement](https://mlsanity.com/paper/2203.15149/) (2022)
- [Analysing Diffusion-based Generative Approaches versus Discriminative Approaches for Speech Restoration](https://mlsanity.com/paper/2211.02397/) (2022)
- [Unsupervised Speech Enhancement using Dynamical Variational Auto-Encoders](https://mlsanity.com/paper/2106.12271/) (2021)
- [DNN-based Source Enhancement to Increase Objective Sound Quality Assessment Score](https://mlsanity.com/paper/1810.09137/) (2018)
- [End-to-End Multi-Task Denoising for joint SDR and PESQ Optimization](https://mlsanity.com/paper/1901.09146/) (2019)
- [Learning with Learned Loss Function: Speech Enhancement with Quality-Net to Improve Perceptual Evaluation of Speech Quality](https://mlsanity.com/paper/1905.01898/) (2019)
- [Deep Noise Suppression Maximizing Non-Differentiable PESQ Mediated by a Non-Intrusive PESQNet](https://mlsanity.com/paper/2111.03847/) (2021)
- [Hallucination in Perceptual Metric-Driven Speech Enhancement Networks](https://mlsanity.com/paper/2403.11732/) (2024)

## Built on by (most cited first, 12 of 12)

- [Robust One-step Speech Enhancement via Consistency Distillation](https://mlsanity.com/paper/2507.05688/) (2025)
- [VoiceBridge: General Speech Restoration with One-step Latent Bridge Models](https://mlsanity.com/paper/2509.25275/) (2025)
- [Investigating the Impact of Speech Enhancement on Audio Deepfake Detection in Noisy Environments](https://mlsanity.com/paper/2603.14767/) (2026)
- [A Hybrid Model for Weakly-Supervised Speech Dereverberation](https://mlsanity.com/paper/2502.06839/) (2025)
- [WhiSQA: Non-Intrusive Speech Quality Prediction Using Whisper Encoder Features](https://mlsanity.com/paper/2508.02210/) (2025)
- [Improving Voice Quality in Speech Anonymization With Just Perception-Informed Losses](https://mlsanity.com/paper/2410.15499/) (2024)
- [Non-intrusive Speech Quality Assessment with Diffusion Models Trained on Clean Speech](https://mlsanity.com/paper/2410.17834/) (2024)
- [Investigating training objective for flow matching-based speech enhancement](https://mlsanity.com/paper/2512.10382/) (2025)
- [Post-Training Speech Enhancement Language Models with Perceptual Rewards](https://mlsanity.com/paper/2606.21458/) (2026)
- [Assessing speech quality metrics for evaluation of neural audio codecs under clean speech conditions](https://mlsanity.com/paper/2509.24457/) (2025)
- [Parallel Time-Band Mixing with Learned Observation-Adding for Robust ASR Front-Ends](https://mlsanity.com/paper/2608.30326/) (2026)
- [Perceptual Quality Loss or Loss of Perceptual Quality?](https://mlsanity.com/paper/2609.35054/) (2026)

## Abstract (arXiv)

To obtain improved speech enhancement models, researchers often focus on increasing performance according to specific instrumental metrics. However, when the same metric is used in a loss function to optimize models, it may be detrimental to aspects that the given metric does not see. The goal of this paper is to illustrate the risk of overfitting a speech enhancement model to the metric used for evaluation. For this, we introduce enhancement models that exploit the widely used PESQ measure. Our "PESQetarian" model achieves 3.82 PESQ on VB-DMD while scoring very poorly in a listening experiment. While the obtained PESQ value of 3.82 would imply "state-of-the-art" PESQ-performance on the VB-DMD benchmark, our examples show that when optimizing w.r.t. a metric, an isolated evaluation on the same metric may be misleading. Instead, other metrics should be included in the evaluation and the resulting performance predictions should be confirmed by listening.