# Weak-Supervised Dysarthria-invariant Features for Spoken Language Understanding using an FHVAE and Adversarial Training

arXiv: https://arxiv.org/abs/2210.13144 · HTML: https://mlsanity.com/paper/2210.13144/

## Key facts

- No citing papers in the map yet (data through 19 Oct 2026).
- Placed in ASR evaluation, bias and atypical speech, one of 11 directions in Speech and audio language models.
- Builds on 19 of its 26 references that are in the map.
- First posted on arXiv 24 Oct 2022.

## Placed in

- Speech and audio language models / [ASR evaluation, bias and atypical speech](https://mlsanity.com/speech/asr-evaluation-bias-and-atypical-speech/)

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications](https://mlsanity.com/paper/2001.06937/) (2020)
- [Unsupervised Learning of Disentangled and Interpretable Representations from Sequential Data](https://mlsanity.com/paper/1709.07902/) (2017)
- [Personalizing ASR for Dysarthric and Accented Speech with Limited Data](https://mlsanity.com/paper/1907.13511/) (2019)
- [Residual Adapters for Parameter-Efficient ASR Adaptation to Atypical and Accented Speech](https://mlsanity.com/paper/2109.06952/) (2021)
- [Adversarial Data Augmentation for Disordered Speech Recognition](https://mlsanity.com/paper/2108.00899/) (2021)
- [Personalized Adversarial Data Augmentation for Dysarthric and Elderly Speech Recognition](https://mlsanity.com/paper/2205.06445/) (2022)
- [Simulating dysarthric speech for training data augmentation in clinical speech applications](https://mlsanity.com/paper/1804.10325/) (2018)
- [Capsule Networks for Low Resource Spoken Language Understanding](https://mlsanity.com/paper/1805.02922/) (2018)
- [A Scalable Model Specialization Framework for Training and Inference using Submodels and its Application to Speech Model Personalization](https://mlsanity.com/paper/2203.12559/) (2022)
- [Scalable Factorized Hierarchical Variational Autoencoder Training](https://mlsanity.com/paper/1804.03201/) (2018)
- [A Study into Pre-training Strategies for Spoken Language Understanding on Dysarthric Speech](https://mlsanity.com/paper/2106.08313/) (2021)
- [Significance of Data Augmentation for Improving Cleft Lip and Palate Speech Recognition](https://mlsanity.com/paper/2110.00797/) (2021)
- [A light transformer for speech-to-intent applications](https://mlsanity.com/paper/2011.12221/) (2020)
- [Speech Disorder Classification Using Extended Factorized Hierarchical Variational Auto-encoders](https://mlsanity.com/paper/2106.07337/) (2021)
- [Unsupervised Representation Learning of Speech for Dialect Identification](https://mlsanity.com/paper/1809.04458/) (2018)

## Abstract (arXiv)

The scarcity of training data and the large speaker variation in dysarthric speech lead to poor accuracy and poor speaker generalization of spoken language understanding systems for dysarthric speech. Through work on the speech features, we focus on improving the model generalization ability with limited dysarthric data. Factorized Hierarchical Variational Auto-Encoders (FHVAE) trained unsupervisedly have shown their advantage in disentangling content and speaker representations. Earlier work showed that the dysarthria shows in both feature vectors. Here, we add adversarial training to bridge the gap between the control and dysarthric speech data domains. We extract dysarthric and speaker invariant features using weak supervision. The extracted features are evaluated on a Spoken Language Understanding task and yield a higher accuracy on unseen speakers with more severe dysarthria compared to features from the basic FHVAE model or plain filterbanks.