# Frequency-Weighted Training Losses for Phoneme-Level DNN-based Speech Enhancement

arXiv: https://arxiv.org/abs/2506.18714 · HTML: https://mlsanity.com/paper/2506.18714/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 30 references that are in the map.
- First posted on arXiv 23 Jun 2025.

## Builds on (most cited first, 11 of 11)

- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [Pyroomacoustics: A Python package for audio room simulations and array processing algorithms](https://mlsanity.com/paper/1710.04196/) (2017)
- [Asteroid: the PyTorch-based audio source separation toolkit for researchers](https://mlsanity.com/paper/2005.04132/) (2020)
- [VERSA: A Versatile Evaluation Toolkit for Speech, Audio, and Music](https://mlsanity.com/paper/2412.17667/) (2024)
- [How Bad Are Artifacts?: Analyzing the Impact of Speech Enhancement Errors on ASR](https://mlsanity.com/paper/2201.06685/) (2022)
- [FaSNet: Low-latency Adaptive Beamforming for Multi-microphone Audio Processing](https://mlsanity.com/paper/1909.13387/) (2019)
- [Joint Training of Speech Enhancement and Self-supervised Model for Noise-robust ASR](https://mlsanity.com/paper/2205.13293/) (2022)
- [DNN-based mask estimation for distributed speech enhancement in spatially unconstrained microphone arrays](https://mlsanity.com/paper/2011.01714/) (2020)
- [DeFTAN-II: Efficient Multichannel Speech Enhancement with Subgroup Processing](https://mlsanity.com/paper/2308.15777/) (2023)
- [Multichannel Speech Enhancement without Beamforming](https://mlsanity.com/paper/2110.13130/) (2021)
- [A Phoneme-Scale Assessment of Multichannel Speech Enhancement Algorithms](https://mlsanity.com/paper/2401.13548/) (2024)

## Built on by (most cited first, 1 of 1)

- [Time-Frequency Weighted Losses for Phoneme Reconstruction in DNN-Based Speech Enhancement](https://mlsanity.com/paper/2606.21635/) (2026)

## Abstract (arXiv)

Recent advances in deep learning have significantly improved multichannel speech enhancement algorithms, yet conventional training loss functions such as the scale-invariant signal-to-distortion ratio (SDR) may fail to preserve fine-grained spectral cues essential for phoneme intelligibility. In this work, we propose perceptually-informed variants of the SDR loss, formulated in the time-frequency domain and modulated by frequency-dependent weighting schemes. These weights are designed to emphasize time-frequency regions where speech is prominent or where the interfering noise is particularly strong. We investigate both fixed and adaptive strategies, including ANSI band-importance weights, spectral magnitude-based weighting, and dynamic weighting based on the relative amount of speech and noise. We train the FaSNet multichannel speech enhancement model using these various losses. Experimental results show that while standard metrics such as the SDR are only marginally improved, their perceptual frequency-weighted counterparts exhibit a more substantial improvement. Besides, spectral and phoneme-level analysis indicates better consonant reconstruction, which points to a better preservation of certain acoustic cues.