# End-to-End Multi-Task Denoising for joint SDR and PESQ Optimization

arXiv: https://arxiv.org/abs/1901.09146 · HTML: https://mlsanity.com/paper/1901.09146/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jan 2019.

## Built on by (most cited first, 8 of 8)

- [MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement](https://mlsanity.com/paper/2104.03538/) (2021)
- [The PESQetarian: On the Relevance of Goodhart's Law for Speech Enhancement](https://mlsanity.com/paper/2406.03460/) (2024)
- [FINALLY: fast and universal speech enhancement with studio-like quality](https://mlsanity.com/paper/2410.05920/) (2024)
- [Robust One-step Speech Enhancement via Consistency Distillation](https://mlsanity.com/paper/2507.05688/) (2025)
- [Speech Boosting: Low-Latency Live Speech Enhancement for TWS Earbuds](https://mlsanity.com/paper/2409.18705/) (2024)
- [Training-Free Multi-Step Audio Source Separation](https://mlsanity.com/paper/2505.19534/) (2025)
- [Quaternion Self-Attention with Shared Scores](https://mlsanity.com/paper/2605.24920/) (2026)
- [Speech Enhancement using Separable Polling Attention and Global Layer Normalization followed with PReLU](https://mlsanity.com/paper/2105.02509/) (2021)

## Abstract (arXiv)

Supervised learning based on a deep neural network recently has achieved substantial improvement on speech enhancement. Denoising networks learn mapping from noisy speech to clean one directly, or to a spectrum mask which is the ratio between clean and noisy spectra. In either case, the network is optimized by minimizing mean square error (MSE) between ground-truth labels and time-domain or spectrum output. However, existing schemes have either of two critical issues: spectrum and metric mismatches. The spectrum mismatch is a well known issue that any spectrum modification after short-time Fourier transform (STFT), in general, cannot be fully recovered after inverse short-time Fourier transform (ISTFT). The metric mismatch is that a conventional MSE metric is sub-optimal to maximize our target metrics, signal-to-distortion ratio (SDR) and perceptual evaluation of speech quality (PESQ). This paper presents a new end-to-end denoising framework with the goal of joint SDR and PESQ optimization. First, the network optimization is performed on the time-domain signals after ISTFT to avoid spectrum mismatch. Second, two loss functions which have improved correlations with SDR and PESQ metrics are proposed to minimize metric mismatch. The experimental result showed that the proposed denoising scheme significantly improved both SDR and PESQ performance over the existing methods.