# Teacher-Student MixIT for Unsupervised and Semi-supervised Speech Separation

arXiv: https://arxiv.org/abs/2106.07843 · HTML: https://mlsanity.com/paper/2106.07843/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Speech and audio distillation, one of 12 directions in Knowledge distillation.
- Builds on 26 of its 37 references that are in the map.
- First posted on arXiv 15 Jun 2021.

## Placed in

- Knowledge distillation / [Speech and audio distillation](https://mlsanity.com/distill/speech-and-audio-distillation/)

## Builds on (most cited first, 26 of 26)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation](https://mlsanity.com/paper/1809.07454/) (2018)
- [Deep clustering: Discriminative embeddings for segmentation and separation](https://mlsanity.com/paper/1508.04306/) (2015)
- [Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separation](https://mlsanity.com/paper/1910.06379/) (2019)
- [WHAM!: Extending Speech Separation to Noisy Environments](https://mlsanity.com/paper/1907.01160/) (2019)
- [Pyroomacoustics: A Python package for audio room simulations and array processing algorithms](https://mlsanity.com/paper/1710.04196/) (2017)
- [Multi-talker Speech Separation with Utterance-level Permutation Invariant Training of Deep Recurrent Neural Networks](https://mlsanity.com/paper/1703.06284/) (2017)
- [The fifth 'CHiME' Speech Separation and Recognition Challenge: Dataset, task and baselines](https://mlsanity.com/paper/1803.10609/) (2018)
- [Wavesplit: End-to-End Speech Separation by Speaker Clustering](https://mlsanity.com/paper/2002.08933/) (2020)
- [WHAMR!: Noisy and Reverberant Single-Channel Speech Separation](https://mlsanity.com/paper/1910.10279/) (2019)
- [Voice Separation with an Unknown Number of Multiple Speakers](https://mlsanity.com/paper/2003.01531/) (2020)
- [Single-Channel Multi-Speaker Separation using Deep Clustering](https://mlsanity.com/paper/1607.02173/) (2016)
- [Differentiable Consistency Constraints for Improved Deep Speech Enhancement](https://mlsanity.com/paper/1811.08521/) (2018)
- [End-to-end Microphone Permutation and Number Invariant Multi-channel Speech Separation](https://mlsanity.com/paper/1910.14104/) (2019)
- [Two-Step Sound Source Separation: Training on Learned Latent Targets](https://mlsanity.com/paper/1910.09804/) (2019)
- [Unsupervised training of a deep clustering model for multichannel blind source separation](https://mlsanity.com/paper/1904.01340/) (2019)
- [SMS-WSJ: Database, performance measures, and baseline recipe for multi-channel source separation and recognition](https://mlsanity.com/paper/1910.13934/) (2019)
- [Unsupervised Deep Clustering for Source Separation: Direct Learning from Mixtures using Spatial Information](https://mlsanity.com/paper/1811.01531/) (2018)
- [Mixup-breakdown: a consistency training method for improving generalization of speech separation models](https://mlsanity.com/paper/1910.13253/) (2019)
- [On End-to-end Multi-channel Time Domain Speech Separation in Reverberant Environments](https://mlsanity.com/paper/2011.05958/) (2020)
- [Deep Bayesian Unsupervised Source Separation Based on a Complex Gaussian Mixture Model](https://mlsanity.com/paper/1908.11307/) (2019)
- [Time-Domain Speech Extraction with Spatial Information and Multi Speaker Conditioning Mechanism](https://mlsanity.com/paper/2102.03762/) (2021)

## Built on by (most cited first, 9 of 9)

- [Music Source Separation with Band-split RNN](https://mlsanity.com/paper/2209.15174/) (2022)
- [RemixIT: Continual self-training of speech enhancement models via bootstrapped remixing](https://mlsanity.com/paper/2202.08862/) (2022)
- [Subnetwork-to-go: Elastic Neural Network with Dynamic Training and Customizable Inference](https://mlsanity.com/paper/2312.03464/) (2023)
- [MixCycle: Unsupervised Speech Separation via Cyclic Mixture Permutation Invariant Training](https://mlsanity.com/paper/2202.03875/) (2022)
- [Continual self-training with bootstrapped remixing for speech enhancement](https://mlsanity.com/paper/2110.10103/) (2021)
- [Deep neural network techniques for monaural speech enhancement: state of the art analysis](https://mlsanity.com/paper/2212.00369/) (2022)
- [Speech Separation with Pretrained Frontend to Minimize Domain Mismatch](https://mlsanity.com/paper/2411.03085/) (2024)
- [Speech Separation based on Contrastive Learning and Deep Modularization](https://mlsanity.com/paper/2305.10652/) (2023)
- [DISPATCH: Distilling Selective Patches for Speech Enhancement](https://mlsanity.com/paper/2509.15922/) (2025)

## Abstract (arXiv)

In this paper, we introduce a novel semi-supervised learning framework for end-to-end speech separation. The proposed method first uses mixtures of unseparated sources and the mixture invariant training (MixIT) criterion to train a teacher model. The teacher model then estimates separated sources that are used to train a student model with standard permutation invariant training (PIT). The student model can be fine-tuned with supervised data, i.e., paired artificial mixtures and clean speech sources, and further improved via model distillation. Experiments with single and multi channel mixtures show that the teacher-student training resolves the over-separation problem observed in the original MixIT method. Further, the semisupervised performance is comparable to a fully-supervised separation system trained using ten times the amount of supervised data.