mlsanityData to 14 Oct 2026

Multiple Choice Learning for Efficient Speech Separation with Many Speakers

arXiv 2411.18497 ↗First posted 27 Nov 2024cs.SD cs.LG eess.AScited by 3

Not on the map. Why?

Abstract (arXiv)

Training speech separation models in the supervised setting raises a permutation problem: finding the best assignation between the model predictions and the ground truth separated signals. This inherently ambiguous task is customarily solved using Permutation Invariant Training (PIT). In this article, we instead consider using the Multiple Choice Learning (MCL) framework, which was originally introduced to tackle ambiguous tasks. We demonstrate experimentally on the popular WSJ0-mix and LibriMix benchmarks that MCL matches the performances of PIT, while being computationally advantageous. This opens the door to a promising research direction, as MCL can be naturally extended to handle a variable number of speakers, or to tackle speech separation in the unsupervised setting.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 29 of its 40 references that are in the map. First posted on arXiv 27 Nov 2024.

Builds on

29 of 40 references.

References · 29

  1. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,316
  2. 2018SDR - half-baked or well done?cited by 423
  3. 2018Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separationcited by 406
  4. 2017Supervised Speech Separation Based on Deep Learning: An Overviewcited by 216
  5. 2015Deep clustering: Discriminative embeddings for segmentation and separationcited by 212
  6. 2016Permutation Invariant Training of Deep Models for Speaker-Independent Multi-talker Speech Separationcited by 191
  7. 2020Attention is All You Need in Speech Separationcited by 185
  8. 2019Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separationcited by 182
  9. 2018Looking to Listen at the Cocktail Party: A Speaker-Independent Audio-Visual Model for Speech Separationcited by 162
  10. 2020LibriMix: An Open-Source Dataset for Generalizable Speech Separationcited by 155
  11. 2016Learning in an Uncertain World: Representing Ambiguity Through Multiple Hypothesescited by 97
  12. 2016Stochastic Multiple Choice Learning for Training Diverse Deep Ensemblescited by 77
17 more
  1. 2020End-to-End Speaker Diarization for an Unknown Number of Speakers with Encoder-Decoder Based Attractorscited by 71
  2. 2022TF-GridNet: Integrating Full- and Sub-Band Modeling for Speech Separationcited by 63
  3. 2020Voice Separation with an Unknown Number of Multiple Speakerscited by 46
  4. 2019Universal Sound Separationcited by 41
  5. 2023MossFormer: Pushing the Performance Limit of Monaural Speech Separation using Gated Single-Head Transformer with Convolution-Augmented Joint Self-Attentionscited by 27
  6. 2016Deep attractor network for single-microphone speaker separationcited by 26
  7. 2020ESPnet-se: end-to-end speech enhancement and separation toolkit designed for asr integrationcited by 22
  8. 2015Joint Optimization of Masks and Deep Recurrent Neural Networks for Monaural Source Separationcited by 20
  9. 2022SepIt: Approaching a Single Channel Speech Separation Boundcited by 16
  10. 2023TS-SEP: Joint Diarization and Separation Conditioned on Estimated Speaker Embeddingscited by 12
  11. 2023Resilient Multiple Choice Learning: A learned scoring scheme with application to audio scene analysiscited by 12
  12. 2024Annealed Multiple Choice Learning: Overcoming limitations of Winner-takes-all with annealingcited by 12
  13. 2021Many-Speakers Single Channel Speech Separation with Optimal Permutation Trainingcited by 11
  14. 2020Towards Listening to 10 People Simultaneously: An Efficient Permutation Invariant Training of Audio Source Separation Using Sinkhorn's Algorithmcited by 8
  15. 2024Boosting Unknown-number Speaker Separation with Transformer Decoder-based Attractorcited by 6
  16. 2022Deep neural network techniques for monaural speech enhancement: state of the art analysiscited by 3
  17. 2020DsMCL: Dual-Level Stochastic Multiple Choice Learning for Multi-Modal Trajectory Predictioncited by 2

Built on by

Cited by year

Citing papers per year: 2025: 3.

3 papers.

  1. 2025MATPAC++: Enhanced Masked Latent Prediction for Self-Supervised Audio Representation Learningcited by 6
  2. 2025TimePre: Bridging Accuracy, Efficiency, and Stability in Probabilistic Time-Series Forecastingcited by 3
  3. 2025Knowing When to Quit: Probabilistic Early Exits for Speech Separationcited by 2