mlsanityData to 7 Oct 2026

Variational Autoencoder for Speech Enhancement with a Noise-Aware Encoder

arXiv 2102.08706 ↗First posted 17 Feb 2021eess.AS cs.LG cs.SDcited by 28

Not on the map. Why?

Abstract (arXiv)

Recently, a generative variational autoencoder (VAE) has been proposed for speech enhancement to model speech statistics. However, this approach only uses clean speech in the training phase, making the estimation particularly sensitive to noise presence, especially in low signal-to-noise ratios (SNRs). To increase the robustness of the VAE, we propose to include noise information in the training phase by using a noise-aware encoder trained on noisy-clean speech pairs. We evaluate our approach on real recordings of different noisy environments and acoustic conditions using two different noise datasets. We show that our proposed noise-aware VAE outperforms the standard VAE in terms of overall distortion without increasing the number of model parameters. At the same time, we demonstrate that our model is capable of generalizing to unseen noise conditions better than a supervised feedforward deep neural network (DNN). Furthermore, we demonstrate the robustness of the model performance to a reduction of the noisy-clean speech training data size.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 9 of its 27 references that are in the map. First posted on arXiv 17 Feb 2021.

Builds on

9 of 27 references.

References · 9

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2013Auto-Encoding Variational Bayescited by 3,980
  3. 2014Semi-Supervised Learning with Deep Generative Modelscited by 494
  4. 2018SDR - half-baked or well done?cited by 420
  5. 2018A Probabilistic U-Net for Segmentation of Ambiguous Imagescited by 281
  6. 2017Supervised Speech Separation Based on Deep Learning: An Overviewcited by 213
  7. 2017Statistical Speech Enhancement Based on Probabilistic Integration of Variational Autoencoder and Non-Negative Matrix Factorizationcited by 33
  8. 2019A variance modeling framework based on variational autoencoders for speech enhancementcited by 33
  9. 2018Semi-supervised multichannel speech enhancement with variational autoencoders and non-negative matrix factorizationcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 9; 2023: 6; 2024: 3; 2025: 6; 2026: 2.

28 papers.

Sort:

By area:

  1. 2022Speech Enhancement and Dereverberation with Diffusion-based Generative Modelscited by 195
  2. 2022StoRM: A Diffusion-based Stochastic Regeneration Model for Speech Enhancement and Dereverberationcited by 105
  3. 2022Speech Enhancement with Score-Based Generative Models in the Complex STFT Domaincited by 102
  4. 2022Analysing Diffusion-based Generative Approaches versus Discriminative Approaches for Speech Restorationcited by 34
  5. 2022Cold Diffusion for Speech Enhancementcited by 31
  6. 2021Unsupervised Speech Enhancement using Dynamical Variational Auto-Encoderscited by 28
  7. 2025LLaSE-G1: Incentivizing Generalization Capability for LLaMA-based Speech Enhancementcited by 19
  8. 2023A Multi-dimensional Deep Structured State Space Approach to Speech Enhancement Using Small-footprint Modelscited by 9
  9. 2023Noise-aware Speech Enhancement using Diffusion Probabilistic Modelcited by 9
  10. 2021Disentanglement Learning for Variational Autoencoders Applied to Audio-Visual Speech Enhancementcited by 7
  11. 2025Speech Enhancement Using Continuous Embeddings of Neural Audio Codeccited by 7
  12. 2023On the Behavior of Intrusive and Non-intrusive Speech Enhancement Metrics in Predictive and Generative Settingscited by 5
16 more
  1. 2023Variance-Preserving-Based Interpolation Diffusion Models for Speech Enhancementcited by 5
  2. 2025LauraTSE: Target Speaker Extraction using Auto-Regressive Decoder-Only Language Modelscited by 5
  3. 2022Integrating Statistical Uncertainty into Neural Network-Based Speech Enhancementcited by 4
  4. 2025VoiceBridge: General Speech Restoration with One-step Latent Bridge Modelscited by 4
  5. 2022Learning and controlling the source-filter representation of speech with a variational autoencodercited by 2
  6. 2022Fast and efficient speech enhancement with variational autoencoderscited by 2
  7. 2024Pre-training Feature Guided Diffusion Model for Speech Enhancementcited by 2
  8. 2025A Hybrid Model for Weakly-Supervised Speech Dereverberationcited by 2
  9. 2026Rethinking Flow and Diffusion Bridge Models for Speech Enhancementcited by 2
  10. 2024Selective State Space Model for Monaural Speech Enhancementcited by 1
  11. 2022A weighted-variance variational autoencoder model for speech enhancementcited by 0
  12. 2023Integrating Uncertainty into Neural Network-based Speech Enhancementcited by 0
  13. 2023Continuous Modeling of the Denoising Process for Speech Enhancement Based on Deep Learningcited by 0
  14. 2024Extract and Diffuse: Latent Integration for Improved Diffusion-based Speech and Vocal Enhancementcited by 0
  15. 2025Lightweight Wasserstein Audio-Visual Model for Unified Speech Enhancement and Separationcited by 0
  16. 2026SB-RF: Schr"odinger Bridge Rectified Flow for One-Step Robust Speech Enhancementcited by 0