# Listening to Sounds of Silence for Speech Denoising

arXiv: https://arxiv.org/abs/2010.12013 · HTML: https://mlsanity.com/paper/2010.12013/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Oct 2020.

## Built on by (most cited first, 9 of 9)

- [Learning Audio-Visual Dereverberation](https://mlsanity.com/paper/2106.07732/) (2021)
- [I Can Hear You: Selective Robust Training for Deepfake Audio Detection](https://mlsanity.com/paper/2411.00121/) (2024)
- [BirdSoundsDenoising: Deep Visual Audio Denoising for Bird Sounds](https://mlsanity.com/paper/2210.10196/) (2022)
- [Real-Time Neural Voice Camouflage](https://mlsanity.com/paper/2112.07076/) (2021)
- [Rethinking complex-valued deep neural networks for monaural speech enhancement](https://mlsanity.com/paper/2301.04320/) (2023)
- [Recent Advancements in Deep Learning Applications and Methods for Autonomous Navigation: A Comprehensive Review](https://mlsanity.com/paper/2302.11089/) (2023)
- [D4AM: A General Denoising Framework for Downstream Acoustic Models](https://mlsanity.com/paper/2311.16595/) (2023)
- [Computational Charisma -- A Brick by Brick Blueprint for Building Charismatic Artificial Intelligence](https://mlsanity.com/paper/2301.00142/) (2022)
- [Knowledge Distillation for Speech Denoising by Latent Representation Alignment with Cosine Distance](https://mlsanity.com/paper/2505.03442/) (2025)

## Abstract (arXiv)

We introduce a deep learning model for speech denoising, a long-standing challenge in audio analysis arising in numerous applications. Our approach is based on a key observation about human speech: there is often a short pause between each sentence or word. In a recorded speech signal, those pauses introduce a series of time periods during which only noise is present. We leverage these incidental silent intervals to learn a model for automatic speech denoising given only mono-channel audio. Detected silent intervals over time expose not just pure noise but its time-varying features, allowing the model to learn noise dynamics and suppress it from the speech signal. Experiments on multiple datasets confirm the pivotal role of silent interval detection for speech denoising, and our method outperforms several state-of-the-art denoising methods, including those that accept only audio input (like ours) and those that denoise based on audiovisual input (and hence require more information). We also show that our method enjoys excellent generalization properties, such as denoising spoken languages not seen during training.