mlsanityData to 7 Oct 2026

Towards Hierarchical Spoken Language Dysfluency Modeling

arXiv 2401.10015 ↗First posted 18 Jan 2024cs.CL eess.AScited by 15

Not on the map. Why?

Abstract (arXiv)

Speech disfluency modeling is the bottleneck for both speech therapy and language learning. However, there is no effective AI solution to systematically tackle this problem. We solidify the concept of disfluent speech and disfluent speech modeling. We then present Hierarchical Unconstrained Disfluency Modeling (H-UDM) approach, the hierarchical extension of UDM that addresses both disfluency transcription and detection to eliminate the need for extensive manual annotation. Our experimental findings serve as clear evidence of the effectiveness and reliability of the methods we have introduced, encompassing both transcription and detection tasks.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 20 of its 41 references that are in the map. First posted on arXiv 18 Jan 2024.

Builds on

20 of 41 references.

References · 20

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  2. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  3. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  4. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 555
  5. 2023Scaling Speech Technology to 1,000+ Languagescited by 401
  6. 2023WhisperX: Time-Accurate Speech Transcription of Long-Form Audiocited by 320
  7. 2023Google USM: Scaling Automatic Speech Recognition Beyond 100 Languagescited by 228
  8. 2023Scaling Laws for Generative Mixed-Modal Language Modelscited by 131
  9. 2020CTC-Segmentation of Large Corpora for German End-to-end Speech Recognitioncited by 75
  10. 2022Robust Disentangled Variational Speech Representation Learning for Zero-shot Voice Conversioncited by 28
  11. 2023AV-data2vec: Self-supervised Learning of Audio-Visual Speech Representations with Contextualized Target Representationscited by 24
  12. 2023Unconstrained Dysfluency Modeling for Dysfluent Speech Transcription and Detectioncited by 21
8 more
  1. 2023Speaker-Independent Acoustic-to-Articulatory Speech Inversioncited by 15
  2. 2022Deep Neural Convolutive Matrix Factorization for Articulatory Representation Decompositioncited by 12
  3. 2022Enhancing ASR for Stuttered Speech with Limited Data Using Detect and Passcited by 11
  4. 2022Articulatory Representation Learning Via Joint Factor Analysis and Neural Matrix Factorizationcited by 11
  5. 2022Phoneme Segmentation Using Self-Supervised Speech Modelscited by 11
  6. 2022Towards Improved Zero-shot Voice Conversion with Conditional DSVAEcited by 10
  7. 2022NeuFA: Neural Network Based End-to-End Forced Alignment with Bidirectional Attention Mechanismcited by 8
  8. 2023Weakly-supervised forced alignment of disfluent speech using phoneme-level modelingcited by 7

Built on by

Cited by year

Citing papers per year: 2024: 6; 2025: 9.

15 papers.

Sort:

By area:

  1. 2024YOLO-Stutter: End-to-end Region-Wise Speech Dysfluency Detectioncited by 16
  2. 2024SSDM: Scalable Speech Dysfluency Modelingcited by 13
  3. 2024Stutter-Solver: End-to-end Multi-lingual Dysfluency Detectioncited by 8
  4. 2025Analysis and Evaluation of Synthetic Data Generation in Speech Dysfluency Detectioncited by 8
  5. 2024Augmenting Automatic Speech Recognition Models with Disfluency Detectioncited by 6
  6. 2024Time and Tokens: Benchmarking End-to-End Speech Dysfluency Detectioncited by 5
  7. 2025Dysfluent WFST: A Framework for Zero-Shot Speech Dysfluency Transcription and Detectioncited by 5
  8. 2025Seamless Dysfluent Speech Text Alignment for Disordered Speech Analysiscited by 4
  9. 2024Self-supervised Speech Models for Word-Level Stuttered Speech Detectioncited by 3
  10. 2025Smooth Operators: LLMs Translating Imperfect Hints into Disfluency-Rich Transcriptscited by 1
  11. 2025K-Function: Joint Pronunciation Transcription and Feedback for Evaluating Kids Language Functioncited by 1
  12. 2025Enabling Conversational Behavior Reasoning Capabilities in Full-Duplex Speechcited by 1
3 more
  1. 2025Revisiting Rule-Based Stuttering Detection: A Comprehensive Analysis of Interpretable Models for Clinical Applicationscited by 0
  2. 2025A Comparative Study of Controllability, Explainability, and Performance in Dysfluency Detection Modelscited by 0
  3. 2025Deploying UDM Series in Real-Life Stuttered Speech Applications: A Clinical Evaluation Frameworkcited by 0