mlsanityData to 14 Oct 2026

Large Language Models for Dysfluency Detection in Stuttered Speech

arXiv 2406.11025 ↗First posted 16 Jun 2024cs.SD cs.CL eess.AScited by 8

Not on the map. Why?

Abstract (arXiv)

Accurately detecting dysfluencies in spoken language can help to improve the performance of automatic speech and language processing components and support the development of more inclusive speech and language technologies. Inspired by the recent trend towards the deployment of large language models (LLMs) as universal learners and processors of non-lexical inputs, such as audio and video, we approach the task of multi-label dysfluency detection as a language modeling problem. We present hypotheses candidates generated with an automatic speech recognition system and acoustic representations extracted from an audio encoder model to an LLM, and finetune the system to predict dysfluency labels on three datasets containing English and German stuttered speech. The experimental results show that our system effectively combines acoustic and lexical information and achieves competitive results on the multi-label stuttering detection task.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 35 of its 51 references that are in the map. First posted on arXiv 16 Jun 2024.

Builds on

35 of 51 references.

References · 35

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  6. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  7. 2023Mistral 7Bcited by 3,164
  8. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  9. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,304
  10. 2023The Falcon Series of Open Language Modelscited by 535
  11. 2024Large Language Models: A Surveycited by 525
  12. 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 506
23 more
  1. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 501
  2. 2023AudioPaLM: A Large Language Model That Can Speak and Listencited by 343
  3. 2023Pengi: An Audio Language Model for Audio Taskscited by 220
  4. 2023Listen, Think, and Understandcited by 209
  5. 2020Is MAP Decoding All You Need? The Inadequacy of the Mode in Neural Machine Translationcited by 138
  6. 2016Towards better decoding and language model integration in sequence to sequence modelscited by 100
  7. 2021Simple and Effective Zero-shot Cross-lingual Phoneme Recognitioncited by 79
  8. 2019Domain Robustness in Neural Machine Translationcited by 65
  9. 2021Sampling-Based Approximations to Minimum Bayes Risk Decoding for Neural Machine Translationcited by 65
  10. 2021Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translationcited by 56
  11. 2021SEP-28k: A Dataset for Stuttering Event Detection From Podcasts With People Who Stuttercited by 52
  12. 2016Disfluency Detection using a Bidirectional LSTMcited by 31
  13. 2022Detecting Dysfluencies in Stuttering Therapy Using wav2vec 2.0cited by 27
  14. 2020Controllable Time-Delay Transformer for Real-Time Punctuation Prediction and Disfluency Detectioncited by 23
  15. 2024It's Never Too Late: Fusing Acoustic Information into Large Language Models for Automatic Speech Recognitioncited by 22
  16. 2022KSoF: The Kassel State of Fluency Dataset -- A Therapy Centered Dataset of Stutteringcited by 21
  17. 2021Machine Learning for Stuttering Identification: Review, Challenges and Future Directionscited by 18
  18. 2023From User Perceptions to Technical Improvement: Enabling People Who Stutter to Better Use Speech Recognitioncited by 18
  19. 2016Neural Machine Translation by Minimising the Bayes-risk with Respect to Syntactic Translation Latticescited by 17
  20. 2021Analysis and Tuning of a Voice Assistant System for Dysfluent Speechcited by 12
  21. 2022The Influence of Dataset Partitioning on Dysfluency Detection Systemscited by 9
  22. 2023A Stutter Seldom Comes Alone -- Cross-Corpus Stuttering Detection as a Multi-label Problemcited by 9
  23. 2024A Multimodal Approach to Device-Directed Speech Detection with Large Language Modelscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 1.

8 papers.

  1. 2025Analysis and Evaluation of Synthetic Data Generation in Speech Dysfluency Detectioncited by 8
  2. 2025Dysfluent WFST: A Framework for Zero-Shot Speech Dysfluency Transcription and Detectioncited by 5
  3. 2025The Sound of Syntax: Finetuning and Comprehensive Evaluation of Language Models for Speech Pathologycited by 2
  4. 2025Smooth Operators: LLMs Translating Imperfect Hints into Disfluency-Rich Transcriptscited by 1
  5. 2025StutterCut: Uncertainty-Guided Normalised Cut for Dysfluency Segmentationcited by 1
  6. 2025Z-Scores: A Metric for Linguistically Assessing Disfluency Removalcited by 1
  7. 2025On the Difficulty of Token-Level Modeling of Dysfluency and Fluency Shaping Artifactscited by 0
  8. 2026Mitigating Scoring Errors and Compensating for Nonverbal Subtests in Speech-Based Dementia Assessmentcited by 0