# Speech Denoising Without Clean Training Data: A Noise2Noise Approach

arXiv: https://arxiv.org/abs/2104.03838 · HTML: https://mlsanity.com/paper/2104.03838/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 23 references that are in the map.
- First posted on arXiv 8 Apr 2021.

## Builds on (most cited first, 9 of 9)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Noise2Noise: Learning Image Restoration without Clean Data](https://mlsanity.com/paper/1803.04189/) (2018)
- [Empirical Evaluation of Rectified Activations in Convolutional Network](https://mlsanity.com/paper/1505.00853/) (2015)
- [Deep Complex Networks](https://mlsanity.com/paper/1705.09792/) (2017)
- [Phase-aware Speech Enhancement with Deep Complex U-Net](https://mlsanity.com/paper/1903.03107/) (2019)
- [Speech Denoising with Deep Feature Losses](https://mlsanity.com/paper/1806.10522/) (2018)
- [Improving Deep Speech Denoising by Noisy2Noisy Signal Mapping](https://mlsanity.com/paper/1904.12069/) (2019)
- [A Review of Multi-Objective Deep Learning Speech Denoising Methods](https://mlsanity.com/paper/2003.12108/) (2020)
- [Convolutional Neural Networks to Enhance Coded Speech](https://mlsanity.com/paper/1806.09411/) (2018)

## Built on by (most cited first, 17 of 17)

- [Unsupervised Speech Enhancement using Dynamical Variational Auto-Encoders](https://mlsanity.com/paper/2106.12271/) (2021)
- [Diffusion models for audio semantic communication](https://mlsanity.com/paper/2309.07195/) (2023)
- [BirdSoundsDenoising: Deep Visual Audio Denoising for Bird Sounds](https://mlsanity.com/paper/2210.10196/) (2022)
- [Self-Supervised Learning from Noisy and Incomplete Data](https://mlsanity.com/paper/2601.03244/) (2026)
- [Disentanglement in a GAN for Unconditional Speech Synthesis](https://mlsanity.com/paper/2307.01673/) (2023)
- [Complex Image Generation SwinTransformer Network for Audio Denoising](https://mlsanity.com/paper/2310.16109/) (2023)
- [DPATD: Dual-Phase Audio Transformer for Denoising](https://mlsanity.com/paper/2310.19588/) (2023)
- [One-Pot Multi-Frame Denoising](https://mlsanity.com/paper/2302.11544/) (2023)
- [Unsupervised speech enhancement with deep dynamical generative speech and noise models](https://mlsanity.com/paper/2306.07820/) (2023)
- [Ultrasonic Image's Annotation Removal: A Self-supervised Noise2Noise Approach](https://mlsanity.com/paper/2307.04133/) (2023)
- [Advancing Perception in Artificial Intelligence through Principles of Cognitive Science](https://mlsanity.com/paper/2310.08803/) (2023)
- [Diffusion Gaussian Mixture Audio Denoise](https://mlsanity.com/paper/2406.09154/) (2024)
- [Complex Image-Generative Diffusion Transformer for Audio Denoising](https://mlsanity.com/paper/2406.09161/) (2024)
- [Denoising Cosine Similarity: A Theory-Driven Approach for Efficient Representation Learning](https://mlsanity.com/paper/2304.09552/) (2023)
- [aTENNuate: Optimized Real-time Speech Enhancement with Deep SSMs on Raw Audio](https://mlsanity.com/paper/2409.03377/) (2024)
- [SSC-UNet: UNet with Self-Supervised Contrastive Learning for Phonocardiography Noise Reduction](https://mlsanity.com/paper/2601.10735/) (2026)
- [Enabling Unsupervised Training of Deep EEG Denoisers With Intelligent Partitioning](https://mlsanity.com/paper/2605.06724/) (2026)

## Abstract (arXiv)

This paper tackles the problem of the heavy dependence of clean speech data required by deep learning based audio-denoising methods by showing that it is possible to train deep speech denoising networks using only noisy speech samples. Conventional wisdom dictates that in order to achieve good speech denoising performance, there is a requirement for a large quantity of both noisy speech samples and perfectly clean speech samples, resulting in a need for expensive audio recording equipment and extremely controlled soundproof recording studios. These requirements pose significant challenges in data collection, especially in economically disadvantaged regions and for low resource languages. This work shows that speech denoising deep neural networks can be successfully trained utilizing only noisy training audio. Furthermore it is revealed that such training regimes achieve superior denoising performance over conventional training regimes utilizing clean training audio targets, in cases involving complex noise distributions and low Signal-to-Noise ratios (high noise environments). This is demonstrated through experiments studying the efficacy of our proposed approach over both real-world noises and synthetic noises using the 20 layered Deep Complex U-Net architecture.