mlsanityData to 7 Oct 2026

On the Difficulty of Token-Level Modeling of Dysfluency and Fluency Shaping Artifacts

arXiv 2512.02027 ↗First posted 18 Nov 2025eess.AS cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Automatic transcription of stuttered speech remains a challenge, even for modern end-to-end (E2E) automatic speech recognition (ASR) frameworks. Dysfluencies and fluency-shaping artifacts are often overlooked, resulting in non-verbatim transcriptions with limited clinical and research value. We propose a parameter-efficient adaptation method to decode dysfluencies and fluency modifications as special tokens within transcriptions, evaluated on simulated (LibriStutter, English) and natural (KSoF, German) stuttered speech datasets. To mitigate ASR performance disparities and bias towards English, we introduce a multi-step fine-tuning strategy with language-adaptive pretraining. Tokenization analysis further highlights the tokenizer's English-centric bias, which poses challenges for improving performance on German data. Our findings demonstrate the effectiveness of lightweight adaptation techniques for dysfluency-aware ASR while exposing key limitations in multilingual E2E systems.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 19 of its 29 references that are in the map. First posted on arXiv 18 Nov 2025.

Builds on

19 of 29 references.

References · 19

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  3. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  4. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 896
  5. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 455
  6. 2024LoRA Learns Less and Forgets Lesscited by 313
  7. 2024ASR Error Correction using Large Language Modelscited by 23
  8. 2022KSoF: The Kassel State of Fluency Dataset -- A Therapy Centered Dataset of Stutteringcited by 21
  9. 2023Unconstrained Dysfluency Modeling for Dysfluent Speech Transcription and Detectioncited by 21
  10. 2021Disfluency Detection with Unlabeled Data and Small BERT Modelscited by 17
  11. 2021Personalized Automatic Speech Recognition Trained on Small Disordered Speech Datasetscited by 14
  12. 2023Adapting an ASR Foundation Model for Spoken Language Assessmentcited by 14
7 more
  1. 2022Teaching BERT to Wait: Balancing Accuracy and Latency for Streaming Disfluency Detectioncited by 12
  2. 2024Large Language Models for Dysfluency Detection in Stuttered Speechcited by 8
  3. 2021Studying Alignment in a Collaborative Learning Activity via Automatic Methods: The Link Between What We Say and Docited by 6
  4. 2022Streaming Joint Speech Recognition and Disfluency Detectioncited by 6
  5. 2024Augmenting Automatic Speech Recognition Models with Disfluency Detectioncited by 6
  6. 2024Inclusive ASR for Disfluent Speech: Cascaded Large-Scale Self-Supervised Learning with Targeted Fine-Tuning and Data Augmentationcited by 5
  7. 2024A Parameter-efficient Language Extension Framework for Multilingual ASRcited by 3

Built on by

Not cited yet.