# Variational Autoencoder for Personalized Pathological Speech Enhancement

arXiv: https://arxiv.org/abs/2503.14036 · HTML: https://mlsanity.com/paper/2503.14036/

## Key facts

- Cited by 4 papers in the map (data through 19 Oct 2026).
- Placed in ASR evaluation, bias and atypical speech, one of 11 directions in Speech and audio language models.
- Builds on 16 of its 36 references that are in the map.
- First posted on arXiv 18 Mar 2025.

## Placed in

- Speech and audio language models / [ASR evaluation, bias and atypical speech](https://mlsanity.com/speech/asr-evaluation-bias-and-atypical-speech/)

## Builds on (most cited first, 16 of 16)

- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [Speech Enhancement and Dereverberation with Diffusion-based Generative Models](https://mlsanity.com/paper/2208.05830/) (2022)
- [Conditional Diffusion Probabilistic Model for Speech Enhancement](https://mlsanity.com/paper/2202.05256/) (2022)
- [A Recurrent Variational Autoencoder for Speech Enhancement](https://mlsanity.com/paper/1910.10942/) (2019)
- [URGENT Challenge: Universality, Robustness, and Generalizability For Speech Enhancement](https://mlsanity.com/paper/2406.04660/) (2024)
- [Statistical Speech Enhancement Based on Probabilistic Integration of Variational Autoencoder and Non-Negative Matrix Factorization](https://mlsanity.com/paper/1710.11439/) (2017)
- [Variational Autoencoder for Speech Enhancement with a Noise-Aware Encoder](https://mlsanity.com/paper/2102.08706/) (2021)
- [Schr"odinger Bridge for Generative Speech Enhancement](https://mlsanity.com/paper/2407.16074/) (2024)
- [A variance modeling framework based on variational autoencoders for speech enhancement](https://mlsanity.com/paper/1902.01605/) (2019)
- [Diffusion Models for Audio Restoration](https://mlsanity.com/paper/2402.09821/) (2024)
- [Personalized Speech Enhancement: New Models and Comprehensive Evaluation](https://mlsanity.com/paper/2110.09625/) (2021)
- [Test-Time Adaptation Toward Personalized Speech Enhancement: Zero-Shot Learning with Knowledge Distillation](https://mlsanity.com/paper/2105.03544/) (2021)
- [Assessing the Generalization Gap of Learning-Based Speech Enhancement Systems in Noisy and Reverberant Environments](https://mlsanity.com/paper/2309.06183/) (2023)
- [Personalized Speech Enhancement through Self-Supervised Data Augmentation and Purification](https://mlsanity.com/paper/2104.02018/) (2021)
- [Personalized Speech Enhancement Without a Separate Speaker Embedding Model](https://mlsanity.com/paper/2406.09928/) (2024)
- [The Effect of Spoken Language on Speech Enhancement using Self-Supervised Speech Representation Loss Functions](https://mlsanity.com/paper/2307.14502/) (2023)

## Built on by (most cited first, 4 of 4)

- [Influence of Clean Speech Characteristics on Speech Enhancement Performance](https://mlsanity.com/paper/2509.18885/) (2025)
- [Generalizability of Predictive and Generative Speech Enhancement Models to Pathological Speakers](https://mlsanity.com/paper/2509.18890/) (2025)
- [Data Augmentation for Pathological Speech Enhancement](https://mlsanity.com/paper/2602.14671/) (2026)
- [Rethinking Batch Composition: Promoting Feature Diversity for Speech Enhancement](https://mlsanity.com/paper/2610.21803/) (2026)

## Abstract (arXiv)

The generalizability of speech enhancement (SE) models across speaker conditions remains largely unexplored, despite its critical importance for broader applicability. This paper investigates the performance of the hybrid variational autoencoder (VAE)-non-negative matrix factorization (NMF) model for SE, focusing primarily on its generalizability to pathological speakers with Parkinson's disease. We show that VAE models trained on large neurotypical datasets perform poorly on pathological speech. While fine-tuning these pre-trained models with pathological speech improves performance, a performance gap remains between neurotypical and pathological speakers. To address this gap, we propose using personalized SE models derived from fine-tuning pre-trained models with only a few seconds of clean data from each speaker. Our results demonstrate that personalized models considerably enhance performance for all speakers, achieving comparable results for both neurotypical and pathological speakers.