# TAPLoss: A Temporal Acoustic Parameter Loss for Speech Enhancement

arXiv: https://arxiv.org/abs/2302.08088 · HTML: https://mlsanity.com/paper/2302.08088/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 35 references that are in the map.
- First posted on arXiv 16 Feb 2023.

## Builds on (most cited first, 18 of 18)

- [DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressors](https://mlsanity.com/paper/2010.15258/) (2020)
- [Real Time Speech Enhancement in the Waveform Domain](https://mlsanity.com/paper/2006.12847/) (2020)
- [DNSMOS P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressors](https://mlsanity.com/paper/2110.01763/) (2021)
- [SEGAN: Speech Enhancement Generative Adversarial Network](https://mlsanity.com/paper/1703.09452/) (2017)
- [The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Testing Framework, and Challenge Results](https://mlsanity.com/paper/2005.13981/) (2020)
- [MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement](https://mlsanity.com/paper/2104.03538/) (2021)
- [End-to-End Multimodal Emotion Recognition using Deep Neural Networks](https://mlsanity.com/paper/1704.08619/) (2017)
- [FullSubNet: A Full-Band and Sub-Band Fusion Model for Real-Time Single-Channel Speech Enhancement](https://mlsanity.com/paper/2010.15508/) (2020)
- [A scalable noisy speech dataset and online subjective test framework](https://mlsanity.com/paper/1909.08050/) (2019)
- [A Wavenet for Speech Denoising](https://mlsanity.com/paper/1706.07162/) (2017)
- [Speech Denoising with Deep Feature Losses](https://mlsanity.com/paper/1806.10522/) (2018)
- [A Differentiable Perceptual Audio Metric Learned from Just Noticeable Differences](https://mlsanity.com/paper/2001.04460/) (2020)
- [I'm Sorry for Your Loss: Spectrally-Based Audio Distances Are Bad at Pitch](https://mlsanity.com/paper/2012.04572/) (2020)
- [NORESQA: A Framework for Speech Quality Assessment using Non-Matching References](https://mlsanity.com/paper/2109.08125/) (2021)
- [Perceptual Loss based Speech Denoising with an ensemble of Audio Pattern Recognition and Self-Supervised Models](https://mlsanity.com/paper/2010.11860/) (2020)
- [Improving Speech Enhancement through Fine-Grained Speech Characteristics](https://mlsanity.com/paper/2207.00237/) (2022)
- [Perceptual Loss with Recognition Model for Single-Channel Enhancement and Robust ASR](https://mlsanity.com/paper/2112.06068/) (2021)
- [Upsampling layers for music source separation](https://mlsanity.com/paper/2111.11773/) (2021)

## Built on by (most cited first, 2 of 2)

- [Improving Perceptual Quality, Intelligibility, and Acoustics on VoIP Platforms](https://mlsanity.com/paper/2303.09048/) (2023)
- [Psychoacoustic Challenges Of Speech Enhancement On VoIP Platforms](https://mlsanity.com/paper/2310.07161/) (2023)

## Abstract (arXiv)

Speech enhancement models have greatly progressed in recent years, but still show limits in perceptual quality of their speech outputs. We propose an objective for perceptual quality based on temporal acoustic parameters. These are fundamental speech features that play an essential role in various applications, including speaker recognition and paralinguistic analysis. We provide a differentiable estimator for four categories of low-level acoustic descriptors involving: frequency-related parameters, energy or amplitude-related parameters, spectral balance parameters, and temporal features. Unlike prior work that looks at aggregated acoustic parameters or a few categories of acoustic parameters, our temporal acoustic parameter (TAP) loss enables auxiliary optimization and improvement of many fine-grain speech characteristics in enhancement workflows. We show that adding TAPLoss as an auxiliary objective in speech enhancement produces speech with improved perceptual quality and intelligibility. We use data from the Deep Noise Suppression 2020 Challenge to demonstrate that both time-domain models and time-frequency domain models can benefit from our method.