# Sparse Mixture of Local Experts for Efficient Speech Enhancement

arXiv: https://arxiv.org/abs/2005.08128 · HTML: https://mlsanity.com/paper/2005.08128/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 May 2020.

## Built on by (most cited first, 8 of 8)

- [Test-Time Adaptation Toward Personalized Speech Enhancement: Zero-Shot Learning with Knowledge Distillation](https://mlsanity.com/paper/2105.03544/) (2021)
- [Efficient Personalized Speech Enhancement through Self-Supervised Learning](https://mlsanity.com/paper/2104.02017/) (2021)
- [Personalized Speech Enhancement through Self-Supervised Data Augmentation and Purification](https://mlsanity.com/paper/2104.02018/) (2021)
- [Zero-Shot Personalized Speech Enhancement through Speaker-Informed Model Selection](https://mlsanity.com/paper/2105.03542/) (2021)
- [Adaptive Slimming for Scalable and Efficient Speech Enhancement](https://mlsanity.com/paper/2507.04879/) (2025)
- [From Diet to Free Lunch: Estimating Auxiliary Signal Properties using Dynamic Pruning Masks in Speech Enhancement Networks](https://mlsanity.com/paper/2602.10666/) (2026)
- [Inference and Denoise: Causal Inference-based Neural Speech Enhancement](https://mlsanity.com/paper/2211.01189/) (2022)
- [HybridSB-MoE: Dual-Domain Schr"odinger Bridges with Scene-Adaptive Expert Routing for Speech Enhancement](https://mlsanity.com/paper/2608.12715/) (2026)

## Abstract (arXiv)

In this paper, we investigate a deep learning approach for speech denoising through an efficient ensemble of specialist neural networks. By splitting up the speech denoising task into non-overlapping subproblems and introducing a classifier, we are able to improve denoising performance while also reducing computational complexity. More specifically, the proposed model incorporates a gating network which assigns noisy speech signals to an appropriate specialist network based on either speech degradation level or speaker gender. In our experiments, a baseline recurrent network is compared against an ensemble of similarly-designed smaller recurrent networks regulated by the auxiliary gating network. Using stochastically generated batches from a large noisy speech corpus, the proposed model learns to estimate a time-frequency masking matrix based on the magnitude spectrogram of an input mixture signal. Both baseline and specialist networks are trained to estimate the ideal ratio mask, while the gating network is trained to perform subproblem classification. Our findings demonstrate that a fine-tuned ensemble network is able to exceed the speech denoising capabilities of a generalist network, doing so with fewer model parameters.