# Time-Frequency Weighted Losses for Phoneme Reconstruction in DNN-Based Speech Enhancement

arXiv: https://arxiv.org/abs/2606.21635 · HTML: https://mlsanity.com/paper/2606.21635/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 31 references that are in the map.
- First posted on arXiv 19 Jun 2026.

## Builds on (most cited first, 9 of 9)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [Pyroomacoustics: A Python package for audio room simulations and array processing algorithms](https://mlsanity.com/paper/1710.04196/) (2017)
- [Asteroid: the PyTorch-based audio source separation toolkit for researchers](https://mlsanity.com/paper/2005.04132/) (2020)
- [End-to-end Microphone Permutation and Number Invariant Multi-channel Speech Separation](https://mlsanity.com/paper/1910.14104/) (2019)
- [Multichannel Long-Term Streaming Neural Speech Enhancement for Static and Moving Speakers](https://mlsanity.com/paper/2403.07675/) (2024)
- [DNN-based mask estimation for distributed speech enhancement in spatially unconstrained microphone arrays](https://mlsanity.com/paper/2011.01714/) (2020)
- [Inter-channel Conv-TasNet for multichannel speech enhancement](https://mlsanity.com/paper/2111.04312/) (2021)
- [Frequency-Weighted Training Losses for Phoneme-Level DNN-based Speech Enhancement](https://mlsanity.com/paper/2506.18714/) (2025)

## Abstract (arXiv)

Conventional training losses for speech enhancement based on the signal-to-distortion ratio (SDR) treat all time-frequency (TF) regions uniformly, overlooking the fine-grained spectral cues that are relevant to specific phoneme intelligibility. We propose a TF weighting framework that modulates the SDR objective based on local speech presence, speech-to-interference ratio (SIR), and spectral flux. By integrating these factors into a differentiable objective, the framework emphasizes TF bins with high speech-noise competition while also accounting for transient cues such as consonant bursts. Experimental results show that our approach improves objective frequency-weighted enhancement metrics, as well as phoneme recognition accuracy, particularly for consonants. Spectral analysis shows better reconstruction of mid-frequency structures at less adverse SIR.