# Analysis and Tuning of a Voice Assistant System for Dysfluent Speech

arXiv: https://arxiv.org/abs/2106.11759 · HTML: https://mlsanity.com/paper/2106.11759/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 25 references that are in the map.
- First posted on arXiv 18 Jun 2021.

## Builds on (most cited first, 10 of 10)

- [SEP-28k: A Dataset for Stuttering Event Detection From Podcasts With People Who Stutter](https://mlsanity.com/paper/2102.12394/) (2021)
- [Detecting Multiple Speech Disfluencies using a Deep Residual Network with Bidirectional Long Short-Term Memory](https://mlsanity.com/paper/1910.12590/) (2019)
- [SNDCNN: Self-normalizing deep CNNs with scaled exponential linear units for speech recognition](https://mlsanity.com/paper/1910.01992/) (2019)
- [FluentNet: End-to-End Detection of Speech Disfluency with Deep Learning](https://mlsanity.com/paper/2009.11394/) (2020)
- [Error-driven Pruning of Language Models for Virtual Assistants](https://mlsanity.com/paper/2102.07219/) (2021)
- [Improved Robustness to Disfluencies in RNN-Transducer Based Speech Recognition](https://mlsanity.com/paper/2012.06259/) (2020)
- [Identification of primary and collateral tracks in stuttered speech](https://mlsanity.com/paper/2003.01018/) (2020)
- [Stuttering Speech Disfluency Prediction using Explainable Attribution Vectors of Facial Muscle Movements](https://mlsanity.com/paper/2010.01231/) (2020)
- [Active Learning for Domain Classification in a Commercial Spoken Personal Assistant](https://mlsanity.com/paper/1908.11404/) (2019)
- [Learning to Rank Intents in Voice Assistants](https://mlsanity.com/paper/2005.00119/) (2020)

## Built on by (most cited first, 12 of 12)

- [Lost in Transcription: Identifying and Quantifying the Accuracy Biases of Automatic Speech Recognition Systems Against Disfluent Speech](https://mlsanity.com/paper/2405.06150/) (2024)
- [Large Language Models for Dysfluency Detection in Stuttered Speech](https://mlsanity.com/paper/2406.11025/) (2024)
- [Robust Stuttering Detection via Multi-task and Adversarial Learning](https://mlsanity.com/paper/2204.01735/) (2022)
- [Automatic Disfluency Detection from Untranscribed Speech](https://mlsanity.com/paper/2311.00867/) (2023)
- [Advancing Stuttering Detection via Data Augmentation, Class-Balanced Loss and Multi-Contextual Deep Learning](https://mlsanity.com/paper/2302.11343/) (2023)
- [AS-70: A Mandarin stuttered speech dataset for automatic speech recognition and stuttering event detection](https://mlsanity.com/paper/2406.07256/) (2024)
- [Inclusive ASR for Disfluent Speech: Cascaded Large-Scale Self-Supervised Learning with Targeted Fine-Tuning and Data Augmentation](https://mlsanity.com/paper/2406.10177/) (2024)
- [Boli: A dataset for understanding stuttering experience and analyzing stuttered speech](https://mlsanity.com/paper/2501.15877/) (2025)
- [Speech Technology Services for Oral History Research](https://mlsanity.com/paper/2405.02333/) (2024)
- [Unheard in the Digital Age: Rethinking AI Bias and Speech Diversity](https://mlsanity.com/paper/2601.18641/) (2026)
- [Aligning Stuttered-Speech Research with End-User Needs: Scoping Review, Survey, and Guidelines](https://mlsanity.com/paper/2604.20535/) (2026)
- [What Counts as an Error? Dual-Reference Benchmarking for Atypical ASR](https://mlsanity.com/paper/2606.31112/) (2026)

## Abstract (arXiv)

Dysfluencies and variations in speech pronunciation can severely degrade speech recognition performance, and for many individuals with moderate-to-severe speech disorders, voice operated systems do not work. Current speech recognition systems are trained primarily with data from fluent speakers and as a consequence do not generalize well to speech with dysfluencies such as sound or word repetitions, sound prolongations, or audible blocks. The focus of this work is on quantitative analysis of a consumer speech recognition system on individuals who stutter and production-oriented approaches for improving performance for common voice assistant tasks (i.e., "what is the weather?"). At baseline, this system introduces a significant number of insertion and substitution errors resulting in intended speech Word Error Rates (isWER) that are 13.64% worse (absolute) for individuals with fluency disorders. We show that by simply tuning the decoding parameters in an existing hybrid speech recognition system one can improve isWER by 24% (relative) for individuals with fluency disorders. Tuning these parameters translates to 3.6% better domain recognition and 1.7% better intent recognition relative to the default setup for the 18 study participants across all stuttering severities.