# Learning with Learned Loss Function: Speech Enhancement with Quality-Net to Improve Perceptual Evaluation of Speech Quality

arXiv: https://arxiv.org/abs/1905.01898 · HTML: https://mlsanity.com/paper/1905.01898/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 May 2019.

## Built on by (most cited first, 8 of 8)

- [MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement](https://mlsanity.com/paper/2104.03538/) (2021)
- [CDPAM: Contrastive learning for perceptual audio similarity](https://mlsanity.com/paper/2102.05109/) (2021)
- [The PESQetarian: On the Relevance of Goodhart's Law for Speech Enhancement](https://mlsanity.com/paper/2406.03460/) (2024)
- [Blind Restoration of Real-World Audio by 1D Operational GANs](https://mlsanity.com/paper/2212.14618/) (2022)
- [Speech Enhancement for Wake-Up-Word detection in Voice Assistants](https://mlsanity.com/paper/2101.12732/) (2021)
- [NeuroAMP: A Novel End-to-end General Purpose Deep Neural Amplifier for Personalized Hearing Aids](https://mlsanity.com/paper/2502.10822/) (2025)
- [Capacity-Net-Based RIS Precoding Design without Channel Estimation for mmWave MIMO System](https://mlsanity.com/paper/2509.25660/) (2025)
- [DiT-Flow: Speech Enhancement Robust to Multiple Distortions based on Flow Matching in Latent Space and Diffusion Transformers](https://mlsanity.com/paper/2603.21608/) (2026)

## Abstract (arXiv)

Utilizing a human-perception-related objective function to train a speech enhancement model has become a popular topic recently. The main reason is that the conventional mean squared error (MSE) loss cannot represent auditory perception well. One of the typical hu-man-perception-related metrics, which is the perceptual evaluation of speech quality (PESQ), has been proven to provide a high correlation to the quality scores rated by humans. Owing to its complex and non-differentiable properties, however, the PESQ function may not be used to optimize speech enhancement models directly. In this study, we propose optimizing the enhancement model with an approximated PESQ function, which is differentiable and learned from the training data. The experimental results show that the learned surrogate function can guide the enhancement model to further boost the PESQ score (in-crease of 0.18 points compared to the results trained with MSE loss) and maintain the speech intelligibility.