# On Loss Functions for Supervised Monaural Time-Domain Speech Enhancement

arXiv: https://arxiv.org/abs/1909.01019 · HTML: https://mlsanity.com/paper/1909.01019/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Sep 2019.

## Built on by (most cited first, 18 of 18)

- [Diffusion-Based Speech Enhancement with Joint Generative and Predictive Decoders](https://mlsanity.com/paper/2305.10734/) (2023)
- [Perceive and predict: self-supervised speech representation based loss functions for speech enhancement](https://mlsanity.com/paper/2301.04388/) (2023)
- [End-to-End Complex-Valued Multidilated Convolutional Neural Network for Joint Acoustic Echo Cancellation and Noise Suppression](https://mlsanity.com/paper/2110.00745/) (2021)
- [TrustEMG-Net: Using Representation-Masking Transformer with U-Net for Surface Electromyography Enhancement](https://mlsanity.com/paper/2410.03843/) (2024)
- [SAV-SE: Scene-aware Audio-Visual Speech Enhancement with Selective State Space Model](https://mlsanity.com/paper/2411.07751/) (2024)
- [Personalized Speech Enhancement through Self-Supervised Data Augmentation and Purification](https://mlsanity.com/paper/2104.02018/) (2021)
- [Speech enhancement with mixture-of-deep-experts with clean clustering pre-training](https://mlsanity.com/paper/2102.06034/) (2021)
- [BASEN: Time-Domain Brain-Assisted Speech Enhancement Network with Convolutional Cross Attention in Multi-talker Conditions](https://mlsanity.com/paper/2305.09994/) (2023)
- [xLSTM-SENet: xLSTM for Single-Channel Speech Enhancement](https://mlsanity.com/paper/2501.06146/) (2025)
- [Deep neural network techniques for monaural speech enhancement: state of the art analysis](https://mlsanity.com/paper/2212.00369/) (2022)
- [Time-domain Speech Enhancement with Generative Adversarial Learning](https://mlsanity.com/paper/2103.16149/) (2021)
- [MambAttention: Mamba with Multi-Head Attention for Generalizable Single-Channel Speech Enhancement](https://mlsanity.com/paper/2507.00966/) (2025)
- [Towards Intelligibility-Oriented Audio-Visual Speech Enhancement](https://mlsanity.com/paper/2111.09642/) (2021)
- [Leveraging Heteroscedastic Uncertainty in Learning Complex Spectral Mapping for Single-channel Speech Enhancement](https://mlsanity.com/paper/2211.08624/) (2022)
- [Hybrid Real- and Complex-valued Neural Network Architecture](https://mlsanity.com/paper/2504.03497/) (2025)
- [Model as Loss: A Self-Consistent Training Paradigm](https://mlsanity.com/paper/2505.21156/) (2025)
- [Exploring Resolution-Wise Shared Attention in Hybrid Mamba-U-Nets for Improved Cross-Corpus Speech Enhancement](https://mlsanity.com/paper/2510.01958/) (2025)
- [RT-SEMamba: Real-Time Speech Enhancement Mamba via Progressive Knowledge Distillation](https://mlsanity.com/paper/2608.12099/) (2026)

## Abstract (arXiv)

Many deep learning-based speech enhancement algorithms are designed to minimize the mean-square error (MSE) in some transform domain between a predicted and a target speech signal. However, optimizing for MSE does not necessarily guarantee high speech quality or intelligibility, which is the ultimate goal of many speech enhancement algorithms. Additionally, only little is known about the impact of the loss function on the emerging class of time-domain deep learning-based speech enhancement systems. We study how popular loss functions influence the performance of deep learning-based speech enhancement systems. First, we demonstrate that perceptually inspired loss functions might be advantageous if the receiver is the human auditory system. Furthermore, we show that the learning rate is a crucial design parameter even for adaptive gradient-based optimizers, which has been generally overlooked in the literature. Also, we found that waveform matching performance metrics must be used with caution as they in certain situations can fail completely. Finally, we show that a loss function based on scale-invariant signal-to-distortion ratio (SI-SDR) achieves good general performance across a range of popular speech enhancement evaluation metrics, which suggests that SI-SDR is a good candidate as a general-purpose loss function for speech enhancement systems.