# A consolidated view of loss functions for supervised deep learning-based speech enhancement

arXiv: https://arxiv.org/abs/2009.12286 · HTML: https://mlsanity.com/paper/2009.12286/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Sep 2020.

## Built on by (most cited first, 19 of 19)

- [Speech Enhancement and Dereverberation with Diffusion-based Generative Models](https://mlsanity.com/paper/2208.05830/) (2022)
- [Speech Enhancement with Score-Based Generative Models in the Complex STFT Domain](https://mlsanity.com/paper/2203.17004/) (2022)
- [CMGAN: Conformer-based Metric GAN for Speech Enhancement](https://mlsanity.com/paper/2203.15149/) (2022)
- [CMGAN: Conformer-Based Metric-GAN for Monaural Speech Enhancement](https://mlsanity.com/paper/2209.11112/) (2022)
- [SCP-GAN: Self-Correcting Discriminator Optimization for Training Consistency Preserving Metric GAN on Speech Enhancement Tasks](https://mlsanity.com/paper/2210.14474/) (2022)
- [SkipConvGAN: Monaural Speech Dereverberation using Generative Adversarial Networks via Complex Time-Frequency Masking](https://mlsanity.com/paper/2211.12623/) (2022)
- [Integrating Statistical Uncertainty into Neural Network-Based Speech Enhancement](https://mlsanity.com/paper/2203.02288/) (2022)
- [A Composite Predictive-Generative Approach to Monaural Universal Speech Enhancement](https://mlsanity.com/paper/2505.24576/) (2025)
- [A Survey of Deep Learning for Complex Speech Spectrograms](https://mlsanity.com/paper/2505.08694/) (2025)
- [Uncertainty Estimation in Deep Speech Enhancement Using Complex Gaussian Mixture Models](https://mlsanity.com/paper/2212.04831/) (2022)
- [Personalized Speech Enhancement Without a Separate Speaker Embedding Model](https://mlsanity.com/paper/2406.09928/) (2024)
- [Fast-ULCNet: A fast and ultra low complexity network for single-channel speech enhancement](https://mlsanity.com/paper/2601.14925/) (2026)
- [Integrating Uncertainty into Neural Network-based Speech Enhancement](https://mlsanity.com/paper/2305.08744/) (2023)
- [Single-Channel Distance-Based Source Separation for Mobile GPU in Outdoor and Indoor Environments](https://mlsanity.com/paper/2501.03045/) (2025)
- [Model as Loss: A Self-Consistent Training Paradigm](https://mlsanity.com/paper/2505.21156/) (2025)
- [Diff-VS: Efficient Audio-Aware Diffusion U-Net for Vocals Separation](https://mlsanity.com/paper/2604.01120/) (2026)
- [SB-RF: Schr"odinger Bridge Rectified Flow for One-Step Robust Speech Enhancement](https://mlsanity.com/paper/2606.05575/) (2026)
- [Rethinking Language Model-Based Generative Speech Enhancement in the Latent Space of a Neural Audio Codec](https://mlsanity.com/paper/2608.12082/) (2026)
- [Does per-frame early exit pay? A compute-matched study of dynamic depth for on-device speech enhancement](https://mlsanity.com/paper/2609.29867/) (2026)

## Abstract (arXiv)

Deep learning-based speech enhancement for real-time applications recently made large advancements. Due to the lack of a tractable perceptual optimization target, many myths around training losses emerged, whereas the contribution to success of the loss functions in many cases has not been investigated isolated from other factors such as network architecture, features, or training procedures. In this work, we investigate a wide variety of loss spectral functions for a recurrent neural network architecture suitable to operate in online frame-by-frame processing. We relate magnitude-only with phase-aware losses, ratios, correlation metrics, and compressed metrics. Our results reveal that combining magnitude-only with phase-aware objectives always leads to improvements, even when the phase is not enhanced. Furthermore, using compressed spectral values also yields a significant improvement. On the other hand, phase-sensitive improvement is best achieved by linear domain losses such as mean absolute error.