# Statistical Speech Enhancement Based on Probabilistic Integration of Variational Autoencoder and Non-Negative Matrix Factorization

arXiv: https://arxiv.org/abs/1710.11439 · HTML: https://mlsanity.com/paper/1710.11439/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2017.

## Built on by (most cited first, 30 of 33)

- [Speech Enhancement and Dereverberation with Diffusion-based Generative Models](https://mlsanity.com/paper/2208.05830/) (2022)
- [Speech Enhancement with Score-Based Generative Models in the Complex STFT Domain](https://mlsanity.com/paper/2203.17004/) (2022)
- [Variational Autoencoder for Speech Enhancement with a Noise-Aware Encoder](https://mlsanity.com/paper/2102.08706/) (2021)
- [Unsupervised Speech Enhancement using Dynamical Variational Auto-Encoders](https://mlsanity.com/paper/2106.12271/) (2021)
- [RemixIT: Continual self-training of speech enhancement models via bootstrapped remixing](https://mlsanity.com/paper/2202.08862/) (2022)
- [The Ethical Implications of Generative Audio Models: A Systematic Literature Review](https://mlsanity.com/paper/2307.05527/) (2023)
- [Target Speech Extraction with Conditional Diffusion Model](https://mlsanity.com/paper/2308.03987/) (2023)
- [Guided Variational Autoencoder for Speech Enhancement With a Supervised Classifier](https://mlsanity.com/paper/2102.06454/) (2021)
- [SRTNet: Time Domain Speech Enhancement Via Stochastic Refinement](https://mlsanity.com/paper/2210.16805/) (2022)
- [Unsupervised speech enhancement with diffusion-based generative models](https://mlsanity.com/paper/2309.10450/) (2023)
- [Variational Selective Autoencoder: Learning from Partially-Observed Heterogeneous Data](https://mlsanity.com/paper/2102.12679/) (2021)
- [Investigating the Design Space of Diffusion Models for Speech Enhancement](https://mlsanity.com/paper/2312.04370/) (2023)
- [Disentanglement Learning for Variational Autoencoders Applied to Audio-Visual Speech Enhancement](https://mlsanity.com/paper/2105.08970/) (2021)
- [SE-Bridge: Speech Enhancement with Consistent Brownian Bridge](https://mlsanity.com/paper/2305.13796/) (2023)
- [Switching Variational Auto-Encoders for Noise-Agnostic Audio-visual Speech Enhancement](https://mlsanity.com/paper/2102.04144/) (2021)
- [Learning robust speech representation with an articulatory-regularized variational autoencoder](https://mlsanity.com/paper/2104.03204/) (2021)
- [Integrating Statistical Uncertainty into Neural Network-Based Speech Enhancement](https://mlsanity.com/paper/2203.02288/) (2022)
- [Deep neural network techniques for monaural speech enhancement: state of the art analysis](https://mlsanity.com/paper/2212.00369/) (2022)
- [A Survey of Deep Learning for Complex Speech Spectrograms](https://mlsanity.com/paper/2505.08694/) (2025)
- [Learning and controlling the source-filter representation of speech with a variational autoencoder](https://mlsanity.com/paper/2204.07075/) (2022)
- [Audio-visual speech enhancement with a deep Kalman filter generative model](https://mlsanity.com/paper/2211.00988/) (2022)
- [Fast and efficient speech enhancement with variational autoencoders](https://mlsanity.com/paper/2211.02728/) (2022)
- [Sound Source Separation Using Latent Variational Block-Wise Disentanglement](https://mlsanity.com/paper/2402.06683/) (2024)
- [Unsupervised speech enhancement with deep dynamical generative speech and noise models](https://mlsanity.com/paper/2306.07820/) (2023)
- [Selective State Space Model for Monaural Speech Enhancement](https://mlsanity.com/paper/2411.06217/) (2024)
- [DenDrift: A Drift-Aware Algorithm for Host Profiling](https://mlsanity.com/paper/2110.01221/) (2021)
- [A weighted-variance variational autoencoder model for speech enhancement](https://mlsanity.com/paper/2211.00990/) (2022)
- [Probabilistic Modelling of Signal Mixtures with Differentiable Dictionaries](https://mlsanity.com/paper/2211.15439/) (2022)
- [Differentiable Dictionary Search: Integrating Linear Mixing with Deep Non-Linear Modelling for Audio Source Separation](https://mlsanity.com/paper/2211.15524/) (2022)
- [Integrating Uncertainty into Neural Network-based Speech Enhancement](https://mlsanity.com/paper/2305.08744/) (2023)

## Abstract (arXiv)

This paper presents a statistical method of single-channel speech enhancement that uses a variational autoencoder (VAE) as a prior distribution on clean speech. A standard approach to speech enhancement is to train a deep neural network (DNN) to take noisy speech as input and output clean speech. Although this supervised approach requires a very large amount of pair data for training, it is not robust against unknown environments. Another approach is to use non-negative matrix factorization (NMF) based on basis spectra trained on clean speech in advance and those adapted to noise on the fly. This semi-supervised approach, however, causes considerable signal distortion in enhanced speech due to the unrealistic assumption that speech spectrograms are linear combinations of the basis spectra. Replacing the poor linear generative model of clean speech in NMF with a VAE---a powerful nonlinear deep generative model---trained on clean speech, we formulate a unified probabilistic generative model of noisy speech. Given noisy speech as observed data, we can sample clean speech from its posterior distribution. The proposed method outperformed the conventional DNN-based method in unseen noisy environments.