# Stuttering Detection Using Speaker Representations and Self-supervised Contextual Embeddings

arXiv: https://arxiv.org/abs/2306.00689 · HTML: https://mlsanity.com/paper/2306.00689/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 40 references that are in the map.
- First posted on arXiv 1 Jun 2023.

## Builds on (most cited first, 19 of 19)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [Self-Supervised Speech Representation Learning: A Review](https://mlsanity.com/paper/2205.10643/) (2022)
- [Emotion Recognition from Speech Using Wav2vec 2.0 Embeddings](https://mlsanity.com/paper/2104.03502/) (2021)
- [A survey on modern trainable activation functions](https://mlsanity.com/paper/2005.00817/) (2020)
- [SEP-28k: A Dataset for Stuttering Event Detection From Podcasts With People Who Stutter](https://mlsanity.com/paper/2102.12394/) (2021)
- [ECAPA-TDNN Embeddings for Speaker Diarization](https://mlsanity.com/paper/2104.01466/) (2021)
- [Detecting Multiple Speech Disfluencies using a Deep Residual Network with Bidirectional Long Short-Term Memory](https://mlsanity.com/paper/1910.12590/) (2019)
- [Detecting Dysfluencies in Stuttering Therapy Using wav2vec 2.0](https://mlsanity.com/paper/2204.03417/) (2022)
- [StutterNet: Stuttering Detection Using Time Delay Neural Network](https://mlsanity.com/paper/2105.05599/) (2021)
- [Machine Learning for Stuttering Identification: Review, Challenges and Future Directions](https://mlsanity.com/paper/2107.04057/) (2021)
- [Speaker-invariant Affective Representation Learning via Adversarial Training](https://mlsanity.com/paper/1911.01533/) (2019)
- [Robust Stuttering Detection via Multi-task and Adversarial Learning](https://mlsanity.com/paper/2204.01735/) (2022)
- [End-to-End and Self-Supervised Learning for ComParE 2022 Stuttering Sub-Challenge](https://mlsanity.com/paper/2207.10817/) (2022)
- [Advancing Stuttering Detection via Data Augmentation, Class-Balanced Loss and Multi-Contextual Deep Learning](https://mlsanity.com/paper/2302.11343/) (2023)
- [The ACM Multimedia 2022 Computational Paralinguistics Challenge: Vocalisations, Stuttering, Activity, & Mosquitoes](https://mlsanity.com/paper/2205.06799/) (2022)

## Built on by (most cited first, 4 of 4)

- [Impact of Speech Mode in Automatic Pathological Speech Detection](https://mlsanity.com/paper/2406.09968/) (2024)
- [Automatically measuring speech fluency in people with aphasia: first achievements using read-speech data](https://mlsanity.com/paper/2308.04763/) (2023)
- [Selfsupervised learning for pathological speech detection](https://mlsanity.com/paper/2406.02572/) (2024)
- [Virtual Speech Therapist: A Clinician-in-the-Loop AI Speech Therapy Agent for Personalized and Supervised Therapy](https://mlsanity.com/paper/2605.01101/) (2026)

## Abstract (arXiv)

The adoption of advanced deep learning architectures in stuttering detection (SD) tasks is challenging due to the limited size of the available datasets. To this end, this work introduces the application of speech embeddings extracted from pre-trained deep learning models trained on large audio datasets for different tasks. In particular, we explore audio representations obtained using emphasized channel attention, propagation, and aggregation time delay neural network (ECAPA-TDNN) and Wav2Vec2.0 models trained on VoxCeleb and LibriSpeech datasets respectively. After extracting the embeddings, we benchmark with several traditional classifiers, such as the K-nearest neighbour (KNN), Gaussian naive Bayes, and neural network, for the SD tasks. In comparison to the standard SD systems trained only on the limited SEP-28k dataset, we obtain a relative improvement of 12.08%, 28.71%, 37.9% in terms of unweighted average recall (UAR) over the baselines. Finally, we have shown that combining two embeddings and concatenating multiple layers of Wav2Vec2.0 can further improve the UAR by up to 2.60% and 6.32% respectively.