mlsanityData to 7 Oct 2026

Voice Separation with an Unknown Number of Multiple Speakers

arXiv 2003.01531 ↗First posted 29 Feb 2020eess.AS cs.LG cs.SDcited by 46

Not on the map. Why?

Abstract (arXiv)

We present a new method for separating a mixed audio sequence, in which multiple voices speak simultaneously. The new method employs gated neural networks that are trained to separate the voices at multiple processing steps, while maintaining the speaker in each output channel fixed. A different model is trained for every number of possible speakers, and the model with the largest number of speakers is employed to select the actual number of speakers in a given sample. Our method greatly outperforms the current state of the art, which, as we show, is not competitive for more than two speakers.

Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 19; 2022: 15; 2023: 9; 2024: 3.

46 papers.

Sort:

By area:

  1. 2022High Fidelity Neural Audio Compressioncited by 673
  2. 2021VisualVoice: Audio-Visual Speech Separation with Cross-Modal Consistencycited by 114
  3. 2024Dual-path Mamba: Short and Long-term Bidirectional Selective Structured State Space Models for Speech Separationcited by 45
  4. 2024MambaMixer: Efficient Selective State Space Models with Dual Token and Channel Selectioncited by 40
  5. 2021The Right to Talk: An Audio-Visual Transformer Approachcited by 33
  6. 2023MossFormer: Pushing the Performance Limit of Monaural Speech Separation using Gated Single-Head Transformer with Convolution-Augmented Joint Self-Attentionscited by 27
  7. 2022Audio-Visual Speech Codecs: Rethinking Audio-Visual Speech Enhancement by Re-Synthesiscited by 24
  8. 2022SepIt: Approaching a Single Channel Speech Separation Boundcited by 16
  9. 2021Towards Neural Diarization for Unlimited Numbers of Speakers Using Global and Local Attractorscited by 14
  10. 2023Separate And Diffuse: Using a Pretrained Diffusion Model for Improving Source Separationcited by 14
  11. 2021Many-Speakers Single Channel Speech Separation with Optimal Permutation Trainingcited by 11
  12. 2021Sandglasset: A Light Multi-Granularity Self-attentive Network For Time-Domain Speech Separationcited by 10
34 more
  1. 2022OCD: Learning to Overfit with Conditional Diffusion Modelscited by 9
  2. 2021Personalized Keyphrase Detection using Speaker and Environment Informationcited by 8
  3. 2021Separate but Together: Unsupervised Federated Learning for Speech Enhancement from Non-IID Datacited by 8
  4. 2021Teacher-Student MixIT for Unsupervised and Semi-supervised Speech Separationcited by 8
  5. 2022ESPnet-SE++: Speech Enhancement for Robust Speech Recognition, Translation, and Understandingcited by 8
  6. 2022ReVISE: Self-Supervised Speech Resynthesis with Visual Input for Universal and Generalized Speech Enhancementcited by 8
  7. 2021REAL-M: Towards Speech Separation on Real Mixturescited by 7
  8. 2022MixCycle: Unsupervised Speech Separation via Cyclic Mixture Permutation Invariant Trainingcited by 7
  9. 2023A Neural State-Space Model Approach to Efficient Speech Separationcited by 6
  10. 2021Directed Speech Separation for Automatic Speech Recognition of Long Form Conversational Speechcited by 5
  11. 2021On permutation invariant training for speech source separationcited by 4
  12. 2021Stepwise-Refining Speech Separation Network via Fine-Grained Encoding in High-order Latent Domaincited by 4
  13. 2022Exploring Self-Attention Mechanisms for Speech Separationcited by 4
  14. 2022Learning to Separate Voices by Spatial Regionscited by 4
  15. 2023UNSSOR: Unsupervised Neural Speech Separation by Leveraging Over-determined Training Mixturescited by 4
  16. 2022MedleyVox: An Evaluation Dataset for Multiple Singing Voices Separationcited by 3
  17. 2022Deep neural network techniques for monaural speech enhancement: state of the art analysiscited by 3
  18. 2024Multiple Choice Learning for Efficient Speech Separation with Many Speakerscited by 3
  19. 2021Time-Domain Speech Extraction with Spatial Information and Multi Speaker Conditioning Mechanismcited by 2
  20. 2021Tune-In: Training Under Negative Environments with Interference for Attention Networks Simulating Cocktail Party Effectcited by 2
  21. 2022Deep Audio Waveform Priorcited by 2
  22. 2023Multi-Dimensional and Multi-Scale Modeling for Speech Separation Optimized by Discriminative Learningcited by 2
  23. 2023End-to-End Integration of Speech Separation and Voice Activity Detection for Low-Latency Diarization of Telephone Conversationscited by 2
  24. 2023An Efficient Speech Separation Network Based on Recurrent Fusion Dilated Convolution and Channel Attentioncited by 2
  25. 2021A Time-domain Real-valued Generalized Wiener Filter for Multi-channel Neural Separation Systemscited by 1
  26. 2022Dual-Path Cross-Modal Attention for better Audio-Visual Speech Extractioncited by 1
  27. 2021Manifold-Aware Deep Clustering: Maximizing Angles between Embedding Vectors Based on Regular Simplexcited by 0
  28. 2021Online Self-Attentive Gated RNNs for Real-Time Speaker Separationcited by 0
  29. 2021Blind Source Separation in Polyphonic Music Recordings Using Deep Neural Networks Trained via Policy Gradientscited by 0
  30. 2021Progressive Learning for Stabilizing Label Selection in Speech Separation with Mapping-based Methodcited by 0
  31. 2022Coarse-to-Fine Recursive Speech Separation for Unknown Number of Speakerscited by 0
  32. 2022Individualized Conditioning and Negative Distances for Speaker Separationcited by 0
  33. 2023Speech Separation based on Contrastive Learning and Deep Modularizationcited by 0
  34. 2023Improving Label Assignments Learning by Dynamic Sample Dropout Combined with Layer-wise Optimization in Speech Separationcited by 0