mlsanityData to 7 Oct 2026

Many-Speakers Single Channel Speech Separation with Optimal Permutation Training

arXiv 2104.08955 ↗First posted 18 Apr 2021cs.SD cs.AI cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

Single channel speech separation has experienced great progress in the last few years. However, training neural speech separation for a large number of speakers (e.g., more than 10 speakers) is out of reach for the current methods, which rely on the Permutation Invariant Loss (PIT). In this work, we present a permutation invariant training that employs the Hungarian algorithm in order to train with an O(C3) time complexity, where C is the number of speakers, in comparison to O(C!) of PIT based methods. Furthermore, we present a modified architecture that can handle the increased number of speakers. Our approach separates up to 20 speakers and improves the previous results for large C by a wide margin.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 19 of its 33 references that are in the map. First posted on arXiv 18 Apr 2021.

Builds on

19 of 33 references.

References · 19

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2018Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separationcited by 400
  3. 2015Deep clustering: Discriminative embeddings for segmentation and separationcited by 211
  4. 2016Permutation Invariant Training of Deep Models for Speaker-Independent Multi-talker Speech Separationcited by 189
  5. 2019Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separationcited by 182
  6. 2020LibriMix: An Open-Source Dataset for Generalizable Speech Separationcited by 154
  7. 2017TasNet: time-domain audio separation network for real-time, single-channel speech separationcited by 115
  8. 2017Multi-talker Speech Separation with Utterance-level Permutation Invariant Training of Deep Recurrent Neural Networkscited by 97
  9. 2020Serialized Output Training for End-to-End Overlapped Speech Recognitioncited by 87
  10. 2020Wavesplit: End-to-End Speech Separation by Speaker Clusteringcited by 74
  11. 2020Voice Separation with an Unknown Number of Multiple Speakerscited by 46
  12. 2016Single-Channel Multi-Speaker Separation using Deep Clusteringcited by 45
7 more
  1. 2016Deep attractor network for single-microphone speaker separationcited by 26
  2. 2019FaSNet: Low-latency Adaptive Beamforming for Multi-microphone Audio Processingcited by 23
  3. 2018Speech Dereverberation Using Fully Convolutional Networkscited by 14
  4. 2018Deep Learning Based Phase Reconstruction for Speaker Separation: A Trigonometric Perspectivecited by 13
  5. 2019Probabilistic Permutation Invariant Training for Speech Separationcited by 12
  6. 2018End-to-End Speech Separation with Unfolded Iterative Phase Reconstructioncited by 10
  7. 2020Towards Listening to 10 People Simultaneously: An Efficient Permutation Invariant Training of Audio Source Separation Using Sinkhorn's Algorithmcited by 8

Built on by

Cited by year

Citing papers per year: 2022: 5; 2023: 3; 2024: 3.

11 papers.

  1. 2022SepIt: Approaching a Single Channel Speech Separation Boundcited by 16
  2. 2023Separate And Diffuse: Using a Pretrained Diffusion Model for Improving Source Separationcited by 14
  3. 2023Latent Autoregressive Source Separationcited by 12
  4. 2022OCD: Learning to Overfit with Conditional Diffusion Modelscited by 9
  5. 2024Listen, Chat, and Remix: Text-Guided Soundscape Remixing for Enhanced Auditory Experiencecited by 6
  6. 2022Learning to Separate Voices by Spatial Regionscited by 4
  7. 2022Show Me the Instruments: Musical Instrument Retrieval from Mixture Audiocited by 4
  8. 2022Deep neural network techniques for monaural speech enhancement: state of the art analysiscited by 3
  9. 2024Multiple Choice Learning for Efficient Speech Separation with Many Speakerscited by 3
  10. 2024AV-CrossNet: an Audiovisual Complex Spectral Mapping Network for Speech Separation By Leveraging Narrow- and Cross-Band Modelingcited by 1
  11. 2023Speech Separation based on Contrastive Learning and Deep Modularizationcited by 0