# Guided Variational Autoencoder for Speech Enhancement With a Supervised Classifier

arXiv: https://arxiv.org/abs/2102.06454 · HTML: https://mlsanity.com/paper/2102.06454/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 23 references that are in the map.
- First posted on arXiv 12 Feb 2021.

## Builds on (most cited first, 8 of 8)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Semi-Supervised Learning with Deep Generative Models](https://mlsanity.com/paper/1406.5298/) (2014)
- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation](https://mlsanity.com/paper/1809.07454/) (2018)
- [Statistical Speech Enhancement Based on Probabilistic Integration of Variational Autoencoder and Non-Negative Matrix Factorization](https://mlsanity.com/paper/1710.11439/) (2017)
- [A variance modeling framework based on variational autoencoders for speech enhancement](https://mlsanity.com/paper/1902.01605/) (2019)
- [Joint Optimization of Masks and Deep Recurrent Neural Networks for Monaural Source Separation](https://mlsanity.com/paper/1502.04149/) (2015)

## Built on by (most cited first, 12 of 12)

- [Speech Enhancement and Dereverberation with Diffusion-based Generative Models](https://mlsanity.com/paper/2208.05830/) (2022)
- [Advancements in Generative AI: A Comprehensive Review of GANs, GPT, Autoencoders, Diffusion Model, and Transformers](https://mlsanity.com/paper/2311.10242/) (2023)
- [Unsupervised Speech Enhancement using Dynamical Variational Auto-Encoders](https://mlsanity.com/paper/2106.12271/) (2021)
- [Disentanglement Learning for Variational Autoencoders Applied to Audio-Visual Speech Enhancement](https://mlsanity.com/paper/2105.08970/) (2021)
- [On the Behavior of Intrusive and Non-intrusive Speech Enhancement Metrics in Predictive and Generative Settings](https://mlsanity.com/paper/2306.03014/) (2023)
- [Integrating Statistical Uncertainty into Neural Network-Based Speech Enhancement](https://mlsanity.com/paper/2203.02288/) (2022)
- [Learning and controlling the source-filter representation of speech with a variational autoencoder](https://mlsanity.com/paper/2204.07075/) (2022)
- [Fast and efficient speech enhancement with variational autoencoders](https://mlsanity.com/paper/2211.02728/) (2022)
- [Unsupervised speech enhancement with deep dynamical generative speech and noise models](https://mlsanity.com/paper/2306.07820/) (2023)
- [Integrating Uncertainty into Neural Network-based Speech Enhancement](https://mlsanity.com/paper/2305.08744/) (2023)
- [Posterior sampling algorithms for unsupervised speech enhancement with recurrent variational autoencoder](https://mlsanity.com/paper/2309.10439/) (2023)
- [Template Collapse and Information-Theoretic Limits in Camera rPPG Pulse Morphology Restoration](https://mlsanity.com/paper/2606.03802/) (2026)

## Abstract (arXiv)

Recently, variational autoencoders have been successfully used to learn a probabilistic prior over speech signals, which is then used to perform speech enhancement. However, variational autoencoders are trained on clean speech only, which results in a limited ability of extracting the speech signal from noisy speech compared to supervised approaches. In this paper, we propose to guide the variational autoencoder with a supervised classifier separately trained on noisy speech. The estimated label is a high-level categorical variable describing the speech signal (e.g. speech activity) allowing for a more informed latent distribution compared to the standard variational autoencoder. We evaluate our method with different types of labels on real recordings of different noisy environments. Provided that the label better informs the latent distribution and that the classifier achieves good performance, the proposed approach outperforms the standard variational autoencoder and a conventional neural network-based supervised approach.