# Speech Denoising with Deep Feature Losses

arXiv: https://arxiv.org/abs/1806.10522 · HTML: https://mlsanity.com/paper/1806.10522/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Jun 2018.

## Built on by (most cited first, 26 of 26)

- [SoundStream: An End-to-End Neural Audio Codec](https://mlsanity.com/paper/2107.03312/) (2021)
- [A Study on Speech Enhancement Based on Diffusion Probabilistic Model](https://mlsanity.com/paper/2107.11876/) (2021)
- [Cold Diffusion for Speech Enhancement](https://mlsanity.com/paper/2211.02527/) (2022)
- [Speech Denoising Without Clean Training Data: A Noise2Noise Approach](https://mlsanity.com/paper/2104.03838/) (2021)
- [Speech Denoising in the Waveform Domain with Self-Attention](https://mlsanity.com/paper/2202.07790/) (2022)
- [Self-Supervised Learning for Speech Enhancement through Synthesis](https://mlsanity.com/paper/2211.02542/) (2022)
- [BirdSoundsDenoising: Deep Visual Audio Denoising for Bird Sounds](https://mlsanity.com/paper/2210.10196/) (2022)
- [Perceptual Contrast Stretching on Target Feature for Speech Enhancement](https://mlsanity.com/paper/2203.17152/) (2022)
- [SkipConvGAN: Monaural Speech Dereverberation using Generative Adversarial Networks via Complex Time-Frequency Masking](https://mlsanity.com/paper/2211.12623/) (2022)
- [EMA2S: An End-to-End Multimodal Articulatory-to-Speech System](https://mlsanity.com/paper/2102.03786/) (2021)
- [Deep neural network techniques for monaural speech enhancement: state of the art analysis](https://mlsanity.com/paper/2212.00369/) (2022)
- [Speech Recovery for Real-World Self-powered Intermittent Devices](https://mlsanity.com/paper/2106.05229/) (2021)
- [A Two-stage Complex Network using Cycle-consistent Generative Adversarial Networks for Speech Enhancement](https://mlsanity.com/paper/2109.02011/) (2021)
- [Upsampling layers for music source separation](https://mlsanity.com/paper/2111.11773/) (2021)
- [TAPLoss: A Temporal Acoustic Parameter Loss for Speech Enhancement](https://mlsanity.com/paper/2302.08088/) (2023)
- [CleanUNet 2: A Hybrid Speech Denoising Model on Waveform and Spectrogram](https://mlsanity.com/paper/2309.05975/) (2023)
- [Single-channel speech enhancement using learnable loss mixup](https://mlsanity.com/paper/2312.17255/) (2023)
- [Audio Denoising for Robust Audio Fingerprinting](https://mlsanity.com/paper/2212.11277/) (2022)
- [D4AM: A General Denoising Framework for Downstream Acoustic Models](https://mlsanity.com/paper/2311.16595/) (2023)
- [Improving Voice Quality in Speech Anonymization With Just Perception-Informed Losses](https://mlsanity.com/paper/2410.15499/) (2024)
- [Speech Enhancement using Separable Polling Attention and Global Layer Normalization followed with PReLU](https://mlsanity.com/paper/2105.02509/) (2021)
- [Towards Unsupervised Learning based Denoising of Cyber Physical System Data to Mitigate Security Concerns](https://mlsanity.com/paper/2303.07530/) (2023)
- [aTENNuate: Optimized Real-time Speech Enhancement with Deep SSMs on Raw Audio](https://mlsanity.com/paper/2409.03377/) (2024)
- [Model as Loss: A Self-Consistent Training Paradigm](https://mlsanity.com/paper/2505.21156/) (2025)
- [From Inaudible Inputs to Model Failures: Low-Frequency Safety Risks in LALMs](https://mlsanity.com/paper/2608.09158/) (2026)
- [SSEMG-Net: A Spectrogram-Based Mamba Network for Surface Electromyography Denoising](https://mlsanity.com/paper/2609.11663/) (2026)

## Abstract (arXiv)

We present an end-to-end deep learning approach to denoising speech signals by processing the raw waveform directly. Given input audio containing speech corrupted by an additive background signal, the system aims to produce a processed signal that contains only the speech content. Recent approaches have shown promising results using various deep network architectures. In this paper, we propose to train a fully-convolutional context aggregation network using a deep feature loss. That loss is based on comparing the internal feature activations in a different network, trained for acoustic environment detection and domestic audio tagging. Our approach outperforms the state-of-the-art in objective speech quality metrics and in large-scale perceptual experiments with human listeners. It also outperforms an identical network trained using traditional regression losses. The advantage of the new approach is particularly pronounced for the hardest data with the most intrusive background noise, for which denoising is most needed and most challenging.