mlsanityData to 7 Oct 2026

Wavesplit: End-to-End Speech Separation by Speaker Clustering

arXiv 2002.08933 ↗First posted 20 Feb 2020eess.AS cs.CL cs.LGcited by 74

Not on the map. Why?

Abstract (arXiv)

We introduce Wavesplit, an end-to-end source separation system. From a single mixture, the model infers a representation for each source and then estimates each source signal given the inferred representations. The model is trained to jointly perform both tasks from the raw waveform. Wavesplit infers a set of source representations via clustering, which addresses the fundamental permutation problem of separation. For speech separation, our sequence-wide speaker representations provide a more robust separation of long, challenging recordings compared to prior work. Wavesplit redefines the state-of-the-art on clean mixtures of 2 or 3 speakers (WSJ0-2/3mix), as well as in noisy and reverberated settings (WHAM/WHAMR). We also set a new benchmark on the recent LibriMix dataset. Finally, we show that Wavesplit is also applicable to other domains, by separating fetal and maternal heart rates from a single abdominal electrocardiogram.

Key facts. Cited by 74 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 21; 2022: 14; 2023: 18; 2024: 11; 2025: 9; 2026: 1.

74 papers.

Sort:

By area:

  1. 2024Dual-path Mamba: Short and Long-term Bidirectional Selective Structured State Space Models for Speech Separationcited by 45
  2. 2022Diffusion-based Generative Speech Source Separationcited by 41
  3. 2024MambaMixer: Efficient Selective State Space Models with Dual Token and Channel Selectioncited by 40
  4. 2025SepPrune: Structured Pruning for Efficient Deep Speech Separationcited by 31
  5. 2023MossFormer: Pushing the Performance Limit of Monaural Speech Separation using Gated Single-Head Transformer with Convolution-Augmented Joint Self-Attentionscited by 27
  6. 2022RemixIT: Continual self-training of speech enhancement models via bootstrapped remixingcited by 20
  7. 2022SepIt: Approaching a Single Channel Speech Separation Boundcited by 16
  8. 2021Compute and memory efficient universal sound source separationcited by 15
  9. 2025Orion-MSP: Multi-Scale Sparse Attention for Tabular In-Context Learningcited by 15
  10. 2021Towards Neural Diarization for Unlimited Numbers of Speakers Using Global and Local Attractorscited by 14
  11. 2023Separate And Diffuse: Using a Pretrained Diffusion Model for Improving Source Separationcited by 14
  12. 2021Many-Speakers Single Channel Speech Separation with Optimal Permutation Trainingcited by 11
38 more
  1. 2023IIANet: An Intra- and Inter-Modality Attention Network for Audio-Visual Speech Separationcited by 11
  2. 2021Sandglasset: A Light Multi-Granularity Self-attentive Network For Time-Domain Speech Separationcited by 10
  3. 2021Estimating the electrical power output of industrial devices with end-to-end time-series classification in the presence of label noisecited by 10
  4. 2023Unifying Speech Enhancement and Separation with Gradient Modulation for End-to-End Noise-Robust Speech Separationcited by 10
  5. 2024Noise-robust Speech Separation with Fast Generative Correctioncited by 9
  6. 2021Teacher-Student MixIT for Unsupervised and Semi-supervised Speech Separationcited by 8
  7. 2021Weight, Block or Unit? Exploring Sparsity Tradeoffs for Speech Enhancement on Tiny Neural Acceleratorscited by 8
  8. 2024OmniSep: Unified Omni-Modality Sound Separation with Query-Mixupcited by 8
  9. 2021REAL-M: Towards Speech Separation on Real Mixturescited by 7
  10. 2022MixCycle: Unsupervised Speech Separation via Cyclic Mixture Permutation Invariant Trainingcited by 7
  11. 2022Online Neural Diarization of Unlimited Numbers of Speakers Using Global and Local Attractorscited by 7
  12. 2023X-SepFormer: End-to-end Speaker Extraction Network with Explicit Optimization on Speaker Confusioncited by 6
  13. 2023A Neural State-Space Model Approach to Efficient Speech Separationcited by 6
  14. 2021DIVE: End-to-end Speech Diarization via Iterative Speaker Embeddingcited by 5
  15. 2021Directed Speech Separation for Automatic Speech Recognition of Long Form Conversational Speechcited by 5
  16. 2022Speaker Extraction with Co-Speech Gestures Cuecited by 5
  17. 2024SepMamba: State-space models for speaker separation using Mambacited by 5
  18. 2021On permutation invariant training for speech source separationcited by 4
  19. 2021Points2Sound: From mono to binaural audio using 3D point cloud scenescited by 4
  20. 2021SVSNet: An End-to-end Speaker Voice Similarity Assessment Modelcited by 4
  21. 2021Stepwise-Refining Speech Separation Network via Fine-Grained Encoding in High-order Latent Domaincited by 4
  22. 2022Exploring Self-Attention Mechanisms for Speech Separationcited by 4
  23. 2023UNSSOR: Unsupervised Neural Speech Separation by Leveraging Over-determined Training Mixturescited by 4
  24. 2023Mixture Encoder for Joint Speech Separation and Recognitioncited by 4
  25. 2025SpectroStream: A Versatile Neural Codec for General Audiocited by 4
  26. 2025A Study of the Scale Invariant Signal to Distortion Ratio in Speech Separation with Noisy Referencescited by 4
  27. 2022Disentangling the Impacts of Language and Channel Variability on Speech Separation Networkscited by 3
  28. 2022Deformable Temporal Convolutional Networks for Monaural Noisy Reverberant Speech Separationcited by 3
  29. 2022MedleyVox: An Evaluation Dataset for Multiple Singing Voices Separationcited by 3
  30. 2022Deep neural network techniques for monaural speech enhancement: state of the art analysiscited by 3
  31. 2023On Time Domain Conformer Models for Monaural Speech Separation in Noisy Reverberant Acoustic Environmentscited by 3
  32. 2024Improving Generalization of Speech Separation in Real-World Scenarios: Strategies in Simulation, Optimization, and Evaluationcited by 3
  33. 2021Time-Domain Speech Extraction with Spatial Information and Multi Speaker Conditioning Mechanismcited by 2
  34. 2021Tune-In: Training Under Negative Environments with Interference for Attention Networks Simulating Cocktail Party Effectcited by 2
  35. 2022CasNet: Investigating Channel Robustness for Speech Separationcited by 2
  36. 2023A Multi-Stage Triple-Path Method for Speech Separation in Noisy and Reverberant Environmentscited by 2
  37. 2023Multi-Dimensional and Multi-Scale Modeling for Speech Separation Optimized by Discriminative Learningcited by 2
  38. 2023On Data Sampling Strategies for Training Neural Network Speech Separation Modelscited by 2

50 most cited of 74 All 74 (JSON)