# Improving Speech Enhancement through Fine-Grained Speech Characteristics

arXiv: https://arxiv.org/abs/2207.00237 · HTML: https://mlsanity.com/paper/2207.00237/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 39 references that are in the map.
- First posted on arXiv 1 Jul 2022.

## Builds on (most cited first, 16 of 16)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressors](https://mlsanity.com/paper/2010.15258/) (2020)
- [Real Time Speech Enhancement in the Waveform Domain](https://mlsanity.com/paper/2006.12847/) (2020)
- [SEGAN: Speech Enhancement Generative Adversarial Network](https://mlsanity.com/paper/1703.09452/) (2017)
- [The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Testing Framework, and Challenge Results](https://mlsanity.com/paper/2005.13981/) (2020)
- [MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement](https://mlsanity.com/paper/2104.03538/) (2021)
- [MetricGAN: Generative Adversarial Networks based Black-box Metric Scores Optimization for Speech Enhancement](https://mlsanity.com/paper/1905.04874/) (2019)
- [FullSubNet: A Full-Band and Sub-Band Fusion Model for Real-Time Single-Channel Speech Enhancement](https://mlsanity.com/paper/2010.15508/) (2020)
- [A scalable noisy speech dataset and online subjective test framework](https://mlsanity.com/paper/1909.08050/) (2019)
- [Self-attention encoding and pooling for speaker recognition](https://mlsanity.com/paper/2008.01077/) (2020)
- [Convolutional-Recurrent Neural Networks for Speech Enhancement](https://mlsanity.com/paper/1805.00579/) (2018)
- [I'm Sorry for Your Loss: Spectrally-Based Audio Distances Are Bad at Pitch](https://mlsanity.com/paper/2012.04572/) (2020)
- [Perceptual Loss based Speech Denoising with an ensemble of Audio Pattern Recognition and Self-Supervised Models](https://mlsanity.com/paper/2010.11860/) (2020)
- [DNN-based Source Enhancement to Increase Objective Sound Quality Assessment Score](https://mlsanity.com/paper/1810.09137/) (2018)
- [Perceptual Loss with Recognition Model for Single-Channel Enhancement and Robust ASR](https://mlsanity.com/paper/2112.06068/) (2021)
- [Speech Denoising with Auditory Models](https://mlsanity.com/paper/2011.10706/) (2020)

## Built on by (most cited first, 5 of 5)

- [TAPLoss: A Temporal Acoustic Parameter Loss for Speech Enhancement](https://mlsanity.com/paper/2302.08088/) (2023)
- [PAAPLoss: A Phonetic-Aligned Acoustic Parameter Loss for Speech Enhancement](https://mlsanity.com/paper/2302.08095/) (2023)
- [Psychoacoustic Challenges Of Speech Enhancement On VoIP Platforms](https://mlsanity.com/paper/2310.07161/) (2023)
- [Speech Enhancement for Virtual Meetings on Cellular Networks](https://mlsanity.com/paper/2302.00868/) (2023)
- [Improving Autism Detection with Multimodal Behavioral Analysis](https://mlsanity.com/paper/2509.21352/) (2025)

## Abstract (arXiv)

While deep learning based speech enhancement systems have made rapid progress in improving the quality of speech signals, they can still produce outputs that contain artifacts and can sound unnatural. We propose a novel approach to speech enhancement aimed at improving perceptual quality and naturalness of enhanced signals by optimizing for key characteristics of speech. We first identify key acoustic parameters that have been found to correlate well with voice quality (e.g. jitter, shimmer, and spectral flux) and then propose objective functions which are aimed at reducing the difference between clean speech and enhanced speech with respect to these features. The full set of acoustic features is the extended Geneva Acoustic Parameter Set (eGeMAPS), which includes 25 different attributes associated with perception of speech. Given the non-differentiable nature of these feature computation, we first build differentiable estimators of the eGeMAPS and then use them to fine-tune existing speech enhancement systems. Our approach is generic and can be applied to any existing deep learning based enhancement systems to further improve the enhanced speech signals. Experimental results conducted on the Deep Noise Suppression (DNS) Challenge dataset shows that our approach can improve the state-of-the-art deep learning based enhancement systems.