# Semi-supervised multichannel speech enhancement with variational autoencoders and non-negative matrix factorization

arXiv: https://arxiv.org/abs/1811.06713 · HTML: https://mlsanity.com/paper/1811.06713/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Nov 2018.

## Built on by (most cited first, 14 of 14)

- [A Survey of Sound Source Localization with Deep Learning Methods](https://mlsanity.com/paper/2109.03465/) (2021)
- [Variational Autoencoder for Speech Enhancement with a Noise-Aware Encoder](https://mlsanity.com/paper/2102.08706/) (2021)
- [Unsupervised Speech Enhancement using Dynamical Variational Auto-Encoders](https://mlsanity.com/paper/2106.12271/) (2021)
- [SRTNet: Time Domain Speech Enhancement Via Stochastic Refinement](https://mlsanity.com/paper/2210.16805/) (2022)
- [SE-Bridge: Speech Enhancement with Consistent Brownian Bridge](https://mlsanity.com/paper/2305.13796/) (2023)
- [Semi-supervised source localization in reverberant environments with deep generative modeling](https://mlsanity.com/paper/2101.10636/) (2021)
- [Learning robust speech representation with an articulatory-regularized variational autoencoder](https://mlsanity.com/paper/2104.03204/) (2021)
- [Unsupervised Music Source Separation Using Differentiable Parametric Source Models](https://mlsanity.com/paper/2201.09592/) (2022)
- [A Survey of Deep Learning for Complex Speech Spectrograms](https://mlsanity.com/paper/2505.08694/) (2025)
- [Learning and controlling the source-filter representation of speech with a variational autoencoder](https://mlsanity.com/paper/2204.07075/) (2022)
- [Neural Fast Full-Rank Spatial Covariance Analysis for Blind Source Separation](https://mlsanity.com/paper/2306.10240/) (2023)
- [Partially Adaptive Multichannel Joint Reduction of Ego-noise and Environmental Noise](https://mlsanity.com/paper/2303.15042/) (2023)
- [Unsupervised speech enhancement with deep dynamical generative speech and noise models](https://mlsanity.com/paper/2306.07820/) (2023)
- [A Unified Probabilistic Framework for Dictionary Learning with Parsimonious Activation](https://mlsanity.com/paper/2509.25690/) (2025)

## Abstract (arXiv)

In this paper we address speaker-independent multichannel speech enhancement in unknown noisy environments. Our work is based on a well-established multichannel local Gaussian modeling framework. We propose to use a neural network for modeling the speech spectro-temporal content. The parameters of this supervised model are learned using the framework of variational autoencoders. The noisy recording environment is supposed to be unknown, so the noise spectro-temporal modeling remains unsupervised and is based on non-negative matrix factorization (NMF). We develop a Monte Carlo expectation-maximization algorithm and we experimentally show that the proposed approach outperforms its NMF-based counterpart, where speech is modeled using supervised NMF.