# Speech enhancement with weakly labelled data from AudioSet

arXiv: https://arxiv.org/abs/2102.09971 · HTML: https://mlsanity.com/paper/2102.09971/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Feb 2021.

## Built on by (most cited first, 9 of 9)

- [Separate Anything You Describe](https://mlsanity.com/paper/2308.05037/) (2023)
- [Zero-shot Audio Source Separation through Query-based Learning from Weakly-labeled Data](https://mlsanity.com/paper/2112.07891/) (2021)
- [Separate but Together: Unsupervised Federated Learning for Speech Enhancement from Non-IID Data](https://mlsanity.com/paper/2105.04727/) (2021)
- [BirdSoundsDenoising: Deep Visual Audio Denoising for Bird Sounds](https://mlsanity.com/paper/2210.10196/) (2022)
- [Learning Temporal Resolution in Spectrogram for Audio Classification](https://mlsanity.com/paper/2210.01719/) (2022)
- [Segment-level Metric Learning for Few-shot Bioacoustic Event Detection](https://mlsanity.com/paper/2207.07773/) (2022)
- [Complex Image Generation SwinTransformer Network for Audio Denoising](https://mlsanity.com/paper/2310.16109/) (2023)
- [FlowSep 2: Self-Supervised Flow Matching for Language-Queried Audio Source Separation](https://mlsanity.com/paper/2608.22111/) (2026)
- [Masked Autoencoders as Universal Speech Enhancer](https://mlsanity.com/paper/2602.02413/) (2026)

## Abstract (arXiv)

Speech enhancement is a task to improve the intelligibility and perceptual quality of degraded speech signal. Recently, neural networks based methods have been applied to speech enhancement. However, many neural network based methods require noisy and clean speech pairs for training. We propose a speech enhancement framework that can be trained with large-scale weakly labelled AudioSet dataset. Weakly labelled data only contain audio tags of audio clips, but not the onset or offset times of speech. We first apply pretrained audio neural networks (PANNs) to detect anchor segments that contain speech or sound events in audio clips. Then, we randomly mix two detected anchor segments containing speech and sound events as a mixture, and build a conditional source separation network using PANNs predictions as soft conditions for speech enhancement. In inference, we input a noisy speech signal with the one-hot encoding of "Speech" as a condition to the trained system to predict enhanced speech. Our system achieves a PESQ of 2.28 and an SSNR of 8.75 dB on the VoiceBank-DEMAND dataset, outperforming the previous SEGAN system of 2.16 and 7.73 dB respectively.