# YOLO-Stutter: End-to-end Region-Wise Speech Dysfluency Detection

arXiv: https://arxiv.org/abs/2408.15297 · HTML: https://mlsanity.com/paper/2408.15297/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 34 references that are in the map.
- First posted on arXiv 27 Aug 2024.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech](https://mlsanity.com/paper/2106.06103/) (2021)
- [Scaling Speech Technology to 1,000+ Languages](https://mlsanity.com/paper/2305.13516/) (2023)
- [WhisperX: Time-Accurate Speech Transcription of Long-Form Audio](https://mlsanity.com/paper/2303.00747/) (2023)
- [Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages](https://mlsanity.com/paper/2303.01037/) (2023)
- [ContentVec: An Improved Self-Supervised Speech Representation by Disentangling Speakers](https://mlsanity.com/paper/2204.09224/) (2022)
- [SEP-28k: A Dataset for Stuttering Event Detection From Podcasts With People Who Stutter](https://mlsanity.com/paper/2102.12394/) (2021)
- [NANSY++: Unified Voice Synthesis with Neural Analysis and Synthesis](https://mlsanity.com/paper/2211.09407/) (2022)
- [Robust Disentangled Variational Speech Representation Learning for Zero-shot Voice Conversion](https://mlsanity.com/paper/2203.16705/) (2022)
- [AV-data2vec: Self-supervised Learning of Audio-Visual Speech Representations with Contextualized Target Representations](https://mlsanity.com/paper/2302.06419/) (2023)
- [Unconstrained Dysfluency Modeling for Dysfluent Speech Transcription and Detection](https://mlsanity.com/paper/2312.12810/) (2023)
- [Self-Supervised Models of Speech Infer Universal Articulatory Kinematics](https://mlsanity.com/paper/2310.10788/) (2023)
- [Speaker-Independent Acoustic-to-Articulatory Speech Inversion](https://mlsanity.com/paper/2302.06774/) (2023)
- [Towards Hierarchical Spoken Language Dysfluency Modeling](https://mlsanity.com/paper/2401.10015/) (2024)
- [Deep Neural Convolutive Matrix Factorization for Articulatory Representation Decomposition](https://mlsanity.com/paper/2204.00465/) (2022)
- [Deep Speech Synthesis from MRI-Based Articulatory Representations](https://mlsanity.com/paper/2307.02471/) (2023)
- [Articulatory Representation Learning Via Joint Factor Analysis and Neural Matrix Factorization](https://mlsanity.com/paper/2210.16498/) (2022)
- [Towards Improved Zero-shot Voice Conversion with Conditional DSVAE](https://mlsanity.com/paper/2205.05227/) (2022)
- [Unsupervised TTS Acoustic Modeling for TTS with Conditional Disentangled Sequential VAE](https://mlsanity.com/paper/2206.02512/) (2022)
- [Weakly-supervised forced alignment of disfluent speech using phoneme-level modeling](https://mlsanity.com/paper/2306.00996/) (2023)

## Built on by (most cited first, 16 of 16)

- [SSDM: Scalable Speech Dysfluency Modeling](https://mlsanity.com/paper/2408.16221/) (2024)
- [Stutter-Solver: End-to-end Multi-lingual Dysfluency Detection](https://mlsanity.com/paper/2409.09621/) (2024)
- [Analysis and Evaluation of Synthetic Data Generation in Speech Dysfluency Detection](https://mlsanity.com/paper/2505.22029/) (2025)
- [Whisper Has an Internal Word Aligner](https://mlsanity.com/paper/2509.09987/) (2025)
- [Time and Tokens: Benchmarking End-to-End Speech Dysfluency Detection](https://mlsanity.com/paper/2409.13582/) (2024)
- [Dysfluent WFST: A Framework for Zero-Shot Speech Dysfluency Transcription and Detection](https://mlsanity.com/paper/2505.16351/) (2025)
- [Seamless Dysfluent Speech Text Alignment for Disordered Speech Analysis](https://mlsanity.com/paper/2506.12073/) (2025)
- [Multilingual Dysarthric Speech Assessment Using Universal Phone Recognition and Language-Specific Phonemic Contrast Modeling](https://mlsanity.com/paper/2601.21205/) (2026)
- [Smooth Operators: LLMs Translating Imperfect Hints into Disfluency-Rich Transcripts](https://mlsanity.com/paper/2506.18510/) (2025)
- [K-Function: Joint Pronunciation Transcription and Feedback for Evaluating Kids Language Function](https://mlsanity.com/paper/2507.03043/) (2025)
- [Revisiting Rule-Based Stuttering Detection: A Comprehensive Analysis of Interpretable Models for Clinical Applications](https://mlsanity.com/paper/2508.16681/) (2025)
- [A Comparative Study of Controllability, Explainability, and Performance in Dysfluency Detection Models](https://mlsanity.com/paper/2509.00058/) (2025)
- [Deploying UDM Series in Real-Life Stuttered Speech Applications: A Clinical Evaluation Framework](https://mlsanity.com/paper/2509.14304/) (2025)
- [HASS: Hierarchical Simulation of Logopenic Aphasic Speech for Scalable PPA Detection](https://mlsanity.com/paper/2603.26795/) (2026)
- [Multilingual Stutter Event Detection for English, German, and Mandarin Speech](https://mlsanity.com/paper/2603.26939/) (2026)
- [Aligning Stuttered-Speech Research with End-User Needs: Scoping Review, Survey, and Guidelines](https://mlsanity.com/paper/2604.20535/) (2026)

## Abstract (arXiv)

Dysfluent speech detection is the bottleneck for disordered speech analysis and spoken language learning. Current state-of-the-art models are governed by rule-based systems which lack efficiency and robustness, and are sensitive to template design. In this paper, we propose YOLO-Stutter: a first end-to-end method that detects dysfluencies in a time-accurate manner. YOLO-Stutter takes imperfect speech-text alignment as input, followed by a spatial feature aggregator, and a temporal dependency extractor to perform region-wise boundary and class predictions. We also introduce two dysfluency corpus, VCTK-Stutter and VCTK-TTS, that simulate natural spoken dysfluencies including repetition, block, missing, replacement, and prolongation. Our end-to-end method achieves state-of-the-art performance with a minimum number of trainable parameters for on both simulated data and real aphasia speech. Code and datasets are open-sourced at https://github.com/rorizzz/YOLO-Stutter