# Detecting Dysfluencies in Stuttering Therapy Using wav2vec 2.0

arXiv: https://arxiv.org/abs/2204.03417 · HTML: https://mlsanity.com/paper/2204.03417/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 31 references that are in the map.
- First posted on arXiv 7 Apr 2022.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Emotion Recognition from Speech Using Wav2vec 2.0 Embeddings](https://mlsanity.com/paper/2104.03502/) (2021)
- [Unsupervised Speech Recognition](https://mlsanity.com/paper/2105.11084/) (2021)
- [Multi-task self-supervised learning for Robust Speech Recognition](https://mlsanity.com/paper/2001.09239/) (2020)
- [SEP-28k: A Dataset for Stuttering Event Detection From Podcasts With People Who Stutter](https://mlsanity.com/paper/2102.12394/) (2021)
- [Detecting Multiple Speech Disfluencies using a Deep Residual Network with Bidirectional Long Short-Term Memory](https://mlsanity.com/paper/1910.12590/) (2019)
- [KSoF: The Kassel State of Fluency Dataset -- A Therapy Centered Dataset of Stuttering](https://mlsanity.com/paper/2203.05383/) (2022)
- [StutterNet: Stuttering Detection Using Time Delay Neural Network](https://mlsanity.com/paper/2105.05599/) (2021)
- [Machine Learning for Stuttering Identification: Review, Challenges and Future Directions](https://mlsanity.com/paper/2107.04057/) (2021)
- [Towards Automated Assessment of Stuttering and Stuttering Therapy](https://mlsanity.com/paper/2006.09222/) (2020)

## Built on by (most cited first, 27 of 27)

- [SSDM: Scalable Speech Dysfluency Modeling](https://mlsanity.com/paper/2408.16221/) (2024)
- [A Stutter Seldom Comes Alone -- Cross-Corpus Stuttering Detection as a Multi-label Problem](https://mlsanity.com/paper/2305.19255/) (2023)
- [Large Language Models for Dysfluency Detection in Stuttered Speech](https://mlsanity.com/paper/2406.11025/) (2024)
- [Unveiling Interpretability in Self-Supervised Speech Representations for Parkinson's Diagnosis](https://mlsanity.com/paper/2412.02006/) (2024)
- [Analysis and Evaluation of Synthetic Data Generation in Speech Dysfluency Detection](https://mlsanity.com/paper/2505.22029/) (2025)
- [Automatic Severity Classification of Dysarthric speech by using Self-supervised Model with Multi-task Learning](https://mlsanity.com/paper/2210.15387/) (2022)
- [Automatic Disfluency Detection from Untranscribed Speech](https://mlsanity.com/paper/2311.00867/) (2023)
- [End-to-End and Self-Supervised Learning for ComParE 2022 Stuttering Sub-Challenge](https://mlsanity.com/paper/2207.10817/) (2022)
- [Whisper in Focus: Enhancing Stuttered Speech Classification with Encoder Layer Optimization](https://mlsanity.com/paper/2311.05203/) (2023)
- [AS-70: A Mandarin stuttered speech dataset for automatic speech recognition and stuttering event detection](https://mlsanity.com/paper/2406.07256/) (2024)
- [Time and Tokens: Benchmarking End-to-End Speech Dysfluency Detection](https://mlsanity.com/paper/2409.13582/) (2024)
- [Dysfluent WFST: A Framework for Zero-Shot Speech Dysfluency Transcription and Detection](https://mlsanity.com/paper/2505.16351/) (2025)
- [Stuttering Detection Using Speaker Representations and Self-supervised Contextual Embeddings](https://mlsanity.com/paper/2306.00689/) (2023)
- [Self-supervised Speech Models for Word-Level Stuttered Speech Detection](https://mlsanity.com/paper/2409.10704/) (2024)
- [Stutter-TTS: Controlled Synthesis and Improved Recognition of Stuttered Speech](https://mlsanity.com/paper/2211.09731/) (2022)
- [Boli: A dataset for understanding stuttering experience and analyzing stuttered speech](https://mlsanity.com/paper/2501.15877/) (2025)
- [Unsupervised speech intelligibility assessment with utterance level alignment distance between teacher and learner Wav2Vec-2.0 representations](https://mlsanity.com/paper/2306.08845/) (2023)
- [Umm... With Transformers? Insights from Filled Pause Use across Four Slavic Parliaments](https://mlsanity.com/paper/2607.05964/) (2026)
- [Toward Generalizable Machine Learning Models in Speech, Language, and Hearing Sciences: Estimating Sample Size and Reducing Overfitting](https://mlsanity.com/paper/2308.11197/) (2023)
- [A Comprehensive Rubric for Annotating Pathological Speech](https://mlsanity.com/paper/2404.18851/) (2024)
- [Optimizing Multi-Stuttered Speech Classification: Leveraging Whisper's Encoder for Efficient Parameter Reduction in Automated Assessment](https://mlsanity.com/paper/2406.05784/) (2024)
- [MMSD-Net: Towards Multi-modal Stuttering Detection](https://mlsanity.com/paper/2407.11492/) (2024)
- [SpikeVox: Towards Energy-Efficient Speech Therapy Framework with Spike-driven Generative Language Models](https://mlsanity.com/paper/2510.15566/) (2025)
- [Multilingual Stutter Event Detection for English, German, and Mandarin Speech](https://mlsanity.com/paper/2603.26939/) (2026)
- [Virtual Speech Therapist: A Clinician-in-the-Loop AI Speech Therapy Agent for Personalized and Supervised Therapy](https://mlsanity.com/paper/2605.01101/) (2026)
- [AURA: Uncertainty-Routed Activation Editing for Acoustic Grounding in Speech Foundation Models](https://mlsanity.com/paper/2609.23979/) (2026)
- [From Detection To Anticipation: Continuous Onset Forecasting For Stuttered Speech](https://mlsanity.com/paper/2610.19235/) (2026)

## Abstract (arXiv)

Stuttering is a varied speech disorder that harms an individual's communication ability. Persons who stutter (PWS) often use speech therapy to cope with their condition. Improving speech recognition systems for people with such non-typical speech or tracking the effectiveness of speech therapy would require systems that can detect dysfluencies while at the same time being able to detect speech techniques acquired in therapy. This paper shows that fine-tuning wav2vec 2.0 [1] for the classification of stuttering on a sizeable English corpus containing stuttered speech, in conjunction with multi-task learning, boosts the effectiveness of the general-purpose wav2vec 2.0 features for detecting stuttering in speech; both within and across languages. We evaluate our method on FluencyBank , [2] and the German therapy-centric Kassel State of Fluency (KSoF) [3] dataset by training Support Vector Machine classifiers using features extracted from the finetuned models for six different stuttering-related event types: blocks, prolongations, sound repetitions, word repetitions, interjections, and - specific to therapy - speech modifications. Using embeddings from the fine-tuned models leads to relative classification performance gains up to 27% w.r.t. F1-score.