mlsanityData to 7 Oct 2026

Time and Tokens: Benchmarking End-to-End Speech Dysfluency Detection

arXiv 2409.13582 ↗First posted 20 Sep 2024eess.AS cs.AI cs.SDcited by 5

Not on the map. Why?

Abstract (arXiv)

Speech dysfluency modeling is a task to detect dysfluencies in speech, such as repetition, block, insertion, replacement, and deletion. Most recent advancements treat this problem as a time-based object detection problem. In this work, we revisit this problem from a new perspective: tokenizing dysfluencies and modeling the detection problem as a token-based automatic speech recognition (ASR) problem. We propose rule-based speech and text dysfluency simulators and develop VCTK-token, and then develop a Whisper-like seq2seq architecture to build a new benchmark with decent performance. We also systematically compare our proposed token-based methods with time-based methods, and propose a unified benchmark to facilitate future research endeavors. We open-source these resources for the broader scientific community. The project page is available at https://rorizzz.github.io/

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 14 of its 26 references that are in the map. First posted on arXiv 20 Sep 2024.

Builds on

14 of 26 references.

References · 14

  1. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  3. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 555
  4. 2023Scaling Speech Technology to 1,000+ Languagescited by 401
  5. 2023Google USM: Scaling Automatic Speech Recognition Beyond 100 Languagescited by 228
  6. 2019Detecting Multiple Speech Disfluencies using a Deep Residual Network with Bidirectional Long Short-Term Memorycited by 27
  7. 2022Detecting Dysfluencies in Stuttering Therapy Using wav2vec 2.0cited by 26
  8. 2023Unconstrained Dysfluency Modeling for Dysfluent Speech Transcription and Detectioncited by 21
  9. 2024YOLO-Stutter: End-to-end Region-Wise Speech Dysfluency Detectioncited by 16
  10. 2024Towards Hierarchical Spoken Language Dysfluency Modelingcited by 15
  11. 2022Deep Neural Convolutive Matrix Factorization for Articulatory Representation Decompositioncited by 12
  12. 2023Deep Speech Synthesis from MRI-Based Articulatory Representationscited by 12
2 more
  1. 2022Articulatory Representation Learning Via Joint Factor Analysis and Neural Matrix Factorizationcited by 11
  2. 2020Detection and Evaluation of human and machine generated speech in spoofing attacks on automatic speaker verification systemscited by 6

Built on by

Cited by year

Citing papers per year: 2025: 5.

5 papers.

  1. 2025Analysis and Evaluation of Synthetic Data Generation in Speech Dysfluency Detectioncited by 8
  2. 2025Dysfluent WFST: A Framework for Zero-Shot Speech Dysfluency Transcription and Detectioncited by 5
  3. 2025Seamless Dysfluent Speech Text Alignment for Disordered Speech Analysiscited by 4
  4. 2025EMO-Reasoning: Benchmarking Emotional Reasoning Capabilities in Spoken Dialogue Systemscited by 2
  5. 2025Smooth Operators: LLMs Translating Imperfect Hints into Disfluency-Rich Transcriptscited by 1