mlsanityData to 7 Oct 2026

WHAM!: Extending Speech Separation to Noisy Environments

arXiv 1907.01160 ↗First posted 2 Jul 2019cs.SD cs.CL cs.LGcited by 154

Not on the map. Why?

Abstract (arXiv)

Recent progress in separating the speech signals from multiple overlapping speakers using a single audio channel has brought us closer to solving the cocktail party problem. However, most studies in this area use a constrained problem setup, comparing performance when speakers overlap almost completely, at artificially low sampling rates, and with no external background noise. In this paper, we strive to move the field towards more realistic and challenging scenarios. To that end, we created the WSJ0 Hipster Ambient Mixtures (WHAM!) dataset, consisting of two speaker mixtures from the wsj0-2mix dataset combined with real ambient noise samples. The samples were collected in coffee shops, restaurants, and bars in the San Francisco Bay Area, and are made publicly available. We benchmark various speech separation architectures and objective functions to evaluate their robustness to noise. While separation performance decreases as a result of noise, we still observe substantial gains relative to the noisy signals for most approaches.

Key facts. Cited by 154 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Jul 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 20; 2022: 15; 2023: 19; 2024: 24; 2025: 34; 2026: 42.

154 papers.

Sort:

By area:

  1. 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 404
  2. 2021SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processingcited by 177
  3. 2022SUPERB-SG: Enhanced Speech processing Universal PERformance Benchmark for Semantic and Generative Capabilitiescited by 76
  4. 2024Mamba-360: Survey of State Space Models as Transformer Alternative for Long Sequence Modelling: Methods, Applications, and Challengescited by 63
  5. 2024EARS: An Anechoic Fullband Speech Dataset Benchmarked for Speech Enhancement and Dereverberationcited by 62
  6. 2023Generative Pre-training for Speech with Flow Matchingcited by 55
  7. 2021ESPnet-SLU: Advancing Spoken Language Understanding through ESPnetcited by 53
  8. 2021A Survey of Sound Source Localization with Deep Learning Methodscited by 45
  9. 2022An efficient encoder-decoder architecture with top-down attention for speech separationcited by 38
  10. 2024A Large-Scale Evaluation of Speech Foundation Modelscited by 38
  11. 2025SepPrune: Structured Pruning for Efficient Deep Speech Separationcited by 31
  12. 2023MossFormer: Pushing the Performance Limit of Monaural Speech Separation using Gated Single-Head Transformer with Convolution-Augmented Joint Self-Attentionscited by 27
38 more
  1. 2021Learning Audio-Visual Dereverberationcited by 26
  2. 2024How Should We Extract Discrete Audio Tokens from Self-Supervised Models?cited by 24
  3. 2021Utilizing Self-supervised Representations for MOS Predictioncited by 20
  4. 2022RemixIT: Continual self-training of speech enhancement models via bootstrapped remixingcited by 20
  5. 2024Speech Robust Bench: A Robustness Benchmark For Speech Recognitioncited by 20
  6. 2021STYLER: Style Factor Modeling with Rapidity and Robustness via Speech Decomposition for Expressive and Controllable Neural Text to Speechcited by 18
  7. 2025FocalCodec: Low-Bitrate Speech Coding via Focal Modulation Networkscited by 18
  8. 2025FlowSE: Efficient and High-Quality Speech Enhancement via Flow Matchingcited by 17
  9. 2023Extending Whisper with prompt tuning to target-speaker ASRcited by 14
  10. 2022Resource-Efficient Separation Transformercited by 11
  11. 2024Look Once to Hear: Target Speech Hearing with Noisy Examplescited by 11
  12. 2023Unifying Speech Enhancement and Separation with Gradient Modulation for End-to-End Noise-Robust Speech Separationcited by 10
  13. 2023Ensemble knowledge distillation of self-supervised speech modelscited by 10
  14. 2024Advancing Multi-talker ASR Performance with Large Language Modelscited by 10
  15. 2024TIGER: Time-frequency Interleaved Gain Extraction and Reconstruction for Efficient Speech Separationcited by 10
  16. 2024Noise-robust Speech Separation with Fast Generative Correctioncited by 9
  17. 2025SoloSpeech: Enhancing Intelligibility and Quality in Target Speech Extraction through a Cascaded Generative Pipelinecited by 9
  18. 2021Test-Time Adaptation Toward Personalized Speech Enhancement: Zero-Shot Learning with Knowledge Distillationcited by 8
  19. 2021Separate but Together: Unsupervised Federated Learning for Speech Enhancement from Non-IID Datacited by 8
  20. 2021Teacher-Student MixIT for Unsupervised and Semi-supervised Speech Separationcited by 8
  21. 2021Hybrid Neural Networks for On-device Directional Hearingcited by 8
  22. 2023DDTSE: Discriminative Diffusion Model for Target Speech Extractioncited by 8
  23. 2026AudioSAE: Towards Understanding of Audio-Processing Models with Sparse AutoEncoderscited by 8
  24. 2021Configurable Privacy-Preserving Automatic Speech Recognitioncited by 7
  25. 2021REAL-M: Towards Speech Separation on Real Mixturescited by 7
  26. 2022Improving the Naturalness of Simulated Conversations for End-to-End Neural Diarizationcited by 7
  27. 2024Serialized Speech Information Guidance with Overlapped Encoding Separation for Multi-Speaker Automatic Speech Recognitioncited by 7
  28. 2021Continual self-training with bootstrapped remixing for speech enhancementcited by 6
  29. 2022Latent Iterative Refinement for Modular Source Separationcited by 6
  30. 2023GASS: Generalizing Audio Source Separation with Large-scale Datacited by 6
  31. 2024Learning Source Disentanglement in Neural Audio Codeccited by 6
  32. 2025FlowTSE: Target Speaker Extraction with Flow Matchingcited by 6
  33. 2022Improving generalizability of distilled self-supervised speech processing models under distorted settingscited by 5
  34. 2025AVEX: What Matters for Animal Vocalization Encodingcited by 5
  35. 2025Serialized Output Prompting for Large Language Model-based Multi-Talker Speech Recognitioncited by 5
  36. 2025UniFlow-Audio: Unified Flow Matching for Audio Generation from Omni-Modalitiescited by 5
  37. 2025PASE: Leveraging the Phonological Prior of WavLM for Low-Hallucination Generative Speech Enhancementcited by 5
  38. 2026FlowSE-GRPO: Training Flow Matching Speech Enhancement via Online Reinforcement Learningcited by 5

50 most cited of 154 All 154 (JSON)