# Speech Enhancement using a Deep Mixture of Experts

arXiv: https://arxiv.org/abs/1703.09302 · HTML: https://mlsanity.com/paper/1703.09302/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal MoE models, one of 12 directions in Mixture-of-experts.
- Builds on 5 of its 33 references that are in the map.
- First posted on arXiv 27 Mar 2017.

## Placed in

- Mixture-of-experts / [Multimodal MoE models](https://mlsanity.com/moe/multimodal-moe-models/)

## Builds on (most cited first, 5 of 5)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Learning Factored Representations in a Deep Mixture of Experts](https://mlsanity.com/paper/1312.4314/) (2013)
- [Theano: A Python framework for fast computation of mathematical expressions](https://mlsanity.com/paper/1605.02688/) (2016)
- [A Hybrid Approach for Speech Enhancement Using MoG Model and Neural Network Phoneme Classifier](https://mlsanity.com/paper/1510.07315/) (2015)

## Built on by (most cited first, 4 of 4)

- [Test-Time Adaptation Toward Personalized Speech Enhancement: Zero-Shot Learning with Knowledge Distillation](https://mlsanity.com/paper/2105.03544/) (2021)
- [A Mixture of Expert Based Deep Neural Network for Improved ASR](https://mlsanity.com/paper/2112.01025/) (2021)
- [Team Deep Mixture of Experts for Distributed Power Control](https://mlsanity.com/paper/2007.14147/) (2020)
- [TF-MoE: Time-Frequency Mixture-of-Experts for Efficient Speech Separation](https://mlsanity.com/paper/2606.29575/) (2026)

## Abstract (arXiv)

In this study we present a Deep Mixture of Experts (DMoE) neural-network architecture for single microphone speech enhancement. By contrast to most speech enhancement algorithms that overlook the speech variability mainly caused by phoneme structure, our framework comprises a set of deep neural networks (DNNs), each one of which is an 'expert' in enhancing a given speech type corresponding to a phoneme. A gating DNN determines which expert is assigned to a given speech segment. A speech presence probability (SPP) is then obtained as a weighted average of the expert SPP decisions, with the weights determined by the gating DNN. A soft spectral attenuation, based on the SPP, is then applied to enhance the noisy speech signal. The experts and the gating components of the DMoE network are trained jointly. As part of the training, speech clustering into different subsets is performed in an unsupervised manner. Therefore, unlike previous methods, a phoneme-labeled database is not required for the training procedure. A series of experiments with different noise types verified the applicability of the new algorithm to the task of speech enhancement. The proposed scheme outperforms other schemes that either do not consider phoneme structure or use a simpler training methodology.