mlsanityData to 14 Oct 2026

Teacher-Student MixIT for Unsupervised and Semi-supervised Speech Separation

arXiv 2106.07843 ↗First posted 15 Jun 2021cs.SD cs.CL eess.AScited by 9

On the map: Knowledge distillation → Speech and audio distillation

Abstract (arXiv)

In this paper, we introduce a novel semi-supervised learning framework for end-to-end speech separation. The proposed method first uses mixtures of unseparated sources and the mixture invariant training (MixIT) criterion to train a teacher model. The teacher model then estimates separated sources that are used to train a student model with standard permutation invariant training (PIT). The student model can be fine-tuned with supervised data, i.e., paired artificial mixtures and clean speech sources, and further improved via model distillation. Experiments with single and multi channel mixtures show that the teacher-student training resolves the over-separation problem observed in the original MixIT method. Further, the semisupervised performance is comparable to a fully-supervised separation system trained using ten times the amount of supervised data.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Speech and audio distillation, one of 12 directions in Knowledge distillation. Builds on 26 of its 37 references that are in the map. First posted on arXiv 15 Jun 2021.

Builds on

26 of 37 references.

References · 26

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  4. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,380
  5. 2018SDR - half-baked or well done?cited by 423
  6. 2018Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separationcited by 406
  7. 2015Deep clustering: Discriminative embeddings for segmentation and separationcited by 212
  8. 2019Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separationcited by 182
  9. 2019WHAM!: Extending Speech Separation to Noisy Environmentscited by 156
  10. 2017Pyroomacoustics: A Python package for audio room simulations and array processing algorithmscited by 132
  11. 2017Multi-talker Speech Separation with Utterance-level Permutation Invariant Training of Deep Recurrent Neural Networkscited by 98
  12. 2018The fifth 'CHiME' Speech Separation and Recognition Challenge: Dataset, task and baselinescited by 92
14 more
  1. 2020Wavesplit: End-to-End Speech Separation by Speaker Clusteringcited by 75
  2. 2019WHAMR!: Noisy and Reverberant Single-Channel Speech Separationcited by 61
  3. 2020Voice Separation with an Unknown Number of Multiple Speakerscited by 46
  4. 2016Single-Channel Multi-Speaker Separation using Deep Clusteringcited by 45
  5. 2018Differentiable Consistency Constraints for Improved Deep Speech Enhancementcited by 35
  6. 2019End-to-end Microphone Permutation and Number Invariant Multi-channel Speech Separationcited by 31
  7. 2019Two-Step Sound Source Separation: Training on Learned Latent Targetscited by 17
  8. 2019Unsupervised training of a deep clustering model for multichannel blind source separationcited by 14
  9. 2019SMS-WSJ: Database, performance measures, and baseline recipe for multi-channel source separation and recognitioncited by 13
  10. 2018Unsupervised Deep Clustering for Source Separation: Direct Learning from Mixtures using Spatial Informationcited by 12
  11. 2019Mixup-breakdown: a consistency training method for improving generalization of speech separation modelscited by 8
  12. 2020On End-to-end Multi-channel Time Domain Speech Separation in Reverberant Environmentscited by 7
  13. 2019Deep Bayesian Unsupervised Source Separation Based on a Complex Gaussian Mixture Modelcited by 3
  14. 2021Time-Domain Speech Extraction with Spatial Information and Multi Speaker Conditioning Mechanismcited by 2

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 4; 2023: 2; 2024: 1; 2025: 1.

9 papers.

  1. 2022Music Source Separation with Band-split RNNcited by 70
  2. 2022RemixIT: Continual self-training of speech enhancement models via bootstrapped remixingcited by 20
  3. 2023Subnetwork-to-go: Elastic Neural Network with Dynamic Training and Customizable Inferencecited by 9
  4. 2022MixCycle: Unsupervised Speech Separation via Cyclic Mixture Permutation Invariant Trainingcited by 7
  5. 2021Continual self-training with bootstrapped remixing for speech enhancementcited by 6
  6. 2022Deep neural network techniques for monaural speech enhancement: state of the art analysiscited by 3
  7. 2024Speech Separation with Pretrained Frontend to Minimize Domain Mismatchcited by 1
  8. 2023Speech Separation based on Contrastive Learning and Deep Modularizationcited by 0
  9. 2025DISPATCH: Distilling Selective Patches for Speech Enhancementcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023A Teacher-Student approach for extracting informative speaker embeddings from speech mixtures
  2. 2022Ultra Fast Speech Separation Model with Teacher Student Learning
  3. 2023Separate And Diffuse: Using a Pretrained Diffusion Model for Improving Source Separation
  4. 2022Leveraging Real Conversational Data for Multi-Channel Continuous Speech Separation
  5. 2022Handling Trade-Offs in Speech Separation with Sparsely-Gated Mixture of Experts
  6. 2025Unsupervised Single-Channel Speech Separation with Diffusion under Speaker-Embedding Guidance
  7. 2025Training-Free Multi-Step Audio Source Separation
  8. 2026Unmixing The Crowd: Learning Persistent Speaker Representations from Mixture-Derived Multi-Speaker Embeddings ↗