# WHAM!: Extending Speech Separation to Noisy Environments

arXiv: https://arxiv.org/abs/1907.01160 · HTML: https://mlsanity.com/paper/1907.01160/

## Key facts

- Cited by 156 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jul 2019.

## Built on by (most cited first, 30 of 156)

- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing](https://mlsanity.com/paper/2110.07205/) (2021)
- [SUPERB-SG: Enhanced Speech processing Universal PERformance Benchmark for Semantic and Generative Capabilities](https://mlsanity.com/paper/2203.06849/) (2022)
- [Mamba-360: Survey of State Space Models as Transformer Alternative for Long Sequence Modelling: Methods, Applications, and Challenges](https://mlsanity.com/paper/2404.16112/) (2024)
- [EARS: An Anechoic Fullband Speech Dataset Benchmarked for Speech Enhancement and Dereverberation](https://mlsanity.com/paper/2406.06185/) (2024)
- [Generative Pre-training for Speech with Flow Matching](https://mlsanity.com/paper/2310.16338/) (2023)
- [ESPnet-SLU: Advancing Spoken Language Understanding through ESPnet](https://mlsanity.com/paper/2111.14706/) (2021)
- [A Survey of Sound Source Localization with Deep Learning Methods](https://mlsanity.com/paper/2109.03465/) (2021)
- [A Large-Scale Evaluation of Speech Foundation Models](https://mlsanity.com/paper/2404.09385/) (2024)
- [An efficient encoder-decoder architecture with top-down attention for speech separation](https://mlsanity.com/paper/2209.15200/) (2022)
- [SepPrune: Structured Pruning for Efficient Deep Speech Separation](https://mlsanity.com/paper/2505.12079/) (2025)
- [MossFormer: Pushing the Performance Limit of Monaural Speech Separation using Gated Single-Head Transformer with Convolution-Augmented Joint Self-Attentions](https://mlsanity.com/paper/2302.11824/) (2023)
- [Learning Audio-Visual Dereverberation](https://mlsanity.com/paper/2106.07732/) (2021)
- [How Should We Extract Discrete Audio Tokens from Self-Supervised Models?](https://mlsanity.com/paper/2406.10735/) (2024)
- [Speech Robust Bench: A Robustness Benchmark For Speech Recognition](https://mlsanity.com/paper/2403.07937/) (2024)
- [Utilizing Self-supervised Representations for MOS Prediction](https://mlsanity.com/paper/2104.03017/) (2021)
- [RemixIT: Continual self-training of speech enhancement models via bootstrapped remixing](https://mlsanity.com/paper/2202.08862/) (2022)
- [STYLER: Style Factor Modeling with Rapidity and Robustness via Speech Decomposition for Expressive and Controllable Neural Text to Speech](https://mlsanity.com/paper/2103.09474/) (2021)
- [FocalCodec: Low-Bitrate Speech Coding via Focal Modulation Networks](https://mlsanity.com/paper/2502.04465/) (2025)
- [FlowSE: Efficient and High-Quality Speech Enhancement via Flow Matching](https://mlsanity.com/paper/2505.19476/) (2025)
- [Extending Whisper with prompt tuning to target-speaker ASR](https://mlsanity.com/paper/2312.08079/) (2023)
- [Ensemble knowledge distillation of self-supervised speech models](https://mlsanity.com/paper/2302.12757/) (2023)
- [Resource-Efficient Separation Transformer](https://mlsanity.com/paper/2206.09507/) (2022)
- [Look Once to Hear: Target Speech Hearing with Noisy Examples](https://mlsanity.com/paper/2405.06289/) (2024)
- [TIGER: Time-frequency Interleaved Gain Extraction and Reconstruction for Efficient Speech Separation](https://mlsanity.com/paper/2410.01469/) (2024)
- [Test-Time Adaptation Toward Personalized Speech Enhancement: Zero-Shot Learning with Knowledge Distillation](https://mlsanity.com/paper/2105.03544/) (2021)
- [Unifying Speech Enhancement and Separation with Gradient Modulation for End-to-End Noise-Robust Speech Separation](https://mlsanity.com/paper/2302.11131/) (2023)
- [Advancing Multi-talker ASR Performance with Large Language Models](https://mlsanity.com/paper/2408.17431/) (2024)
- [Teacher-Student MixIT for Unsupervised and Semi-supervised Speech Separation](https://mlsanity.com/paper/2106.07843/) (2021)
- [Noise-robust Speech Separation with Fast Generative Correction](https://mlsanity.com/paper/2406.07461/) (2024)

## Abstract (arXiv)

Recent progress in separating the speech signals from multiple overlapping speakers using a single audio channel has brought us closer to solving the cocktail party problem. However, most studies in this area use a constrained problem setup, comparing performance when speakers overlap almost completely, at artificially low sampling rates, and with no external background noise. In this paper, we strive to move the field towards more realistic and challenging scenarios. To that end, we created the WSJ0 Hipster Ambient Mixtures (WHAM!) dataset, consisting of two speaker mixtures from the wsj0-2mix dataset combined with real ambient noise samples. The samples were collected in coffee shops, restaurants, and bars in the San Francisco Bay Area, and are made publicly available. We benchmark various speech separation architectures and objective functions to evaluate their robustness to noise. While separation performance decreases as a result of noise, we still observe substantial gains relative to the noisy signals for most approaches.