mlsanityData to 7 Oct 2026

SSDM: Scalable Speech Dysfluency Modeling

arXiv 2408.16221 ↗First posted 29 Aug 2024eess.AS cs.AI cs.CLcited by 13

Not on the map. Why?

Abstract (arXiv)

Speech dysfluency modeling is the core module for spoken language learning, and speech therapy. However, there are three challenges. First, current state-of-the-art solutionscitelian2023unconstrained-udm, lian-anumanchipalli-2024-towards-hudm suffer from poor scalability. Second, there is a lack of a large-scale dysfluency corpus. Third, there is not an effective learning framework. In this paper, we propose textitSSDM: Scalable Speech Dysfluency Modeling, which (1) adopts articulatory gestures as scalable forced alignment; (2) introduces connectionist subsequence aligner (CSA) to achieve dysfluency alignment; (3) introduces a large-scale simulated dysfluency corpus called Libri-Dys; and (4) develops an end-to-end system by leveraging the power of large language models (LLMs). We expect SSDM to serve as a standard in the area of dysfluency modeling. Demo is available at urlhttps://berkeley-speech-group.github.io/SSDM/.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 85 of its 139 references that are in the map. First posted on arXiv 29 Aug 2024.

Builds on

85 of 139 references.

References · 85

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  6. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  8. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  9. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  10. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  11. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  12. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
38 more
  1. 2021SoundStream: An End-to-End Neural Audio Codeccited by 756
  2. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
  3. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 718
  4. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  5. 2022High Fidelity Neural Audio Compressioncited by 673
  6. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 639
  7. 2021AST: Audio Spectrogram Transformercited by 634
  8. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 611
  9. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 555
  10. 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 492
  11. 2023Simple and Controllable Music Generationcited by 480
  12. 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 458
  13. 2023High-Fidelity Audio Compression with Improved RVQGANcited by 423
  14. 2023Scaling Speech Technology to 1,000+ Languagescited by 401
  15. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 373
  16. 2023AudioPaLM: A Large Language Model That Can Speak and Listencited by 333
  17. 2022AudioGen: Textually Guided Audio Generationcited by 292
  18. 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 277
  19. 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 269
  20. 2017Soft-DTW: a Differentiable Loss Function for Time-Seriescited by 265
  21. 2019Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Designcited by 258
  22. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 258
  23. 2023Google USM: Scaling Automatic Speech Recognition Beyond 100 Languagescited by 228
  24. 2023Listen, Think, and Understandcited by 206
  25. 2023Seamless: Multilingual Expressive and Streaming Speech Translationcited by 190
  26. 2024Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilitiescited by 183
  27. 2023Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modelingcited by 166
  28. 2022NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Qualitycited by 151
  29. 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Modelscited by 148
  30. 2021DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERTcited by 129
  31. 2023SoundStorm: Efficient Parallel Audio Generationcited by 111
  32. 2022Efficient Self-supervised Learning with Contextualized Target Representations for Vision, Speech and Languagecited by 108
  33. 2021BYOL for Audio: Self-Supervised Learning for General-Purpose Audio Representationcited by 106
  34. 2023Dynamic-SUPERB: Towards A Dynamic, Collaborative, and Comprehensive Instruction-Tuning Benchmark for Speechcited by 99
  35. 2023Joint Audio and Speech Understandingcited by 99
  36. 2023HiFi-Codec: Group-residual Vector quantization for High Fidelity Audio Codeccited by 95
  37. 2021BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognitioncited by 88
  38. 2023VioLA: Unified Codec Language Models for Speech Recognition, Synthesis, and Translationcited by 87

50 most cited of 85 All 85 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 11; 2026: 2.

13 papers.

Sort:

By area:

  1. 2025Direct Retrieval-augmented Optimization: Synergizing Knowledge Selection and Language Modelscited by 9
  2. 2025Analysis and Evaluation of Synthetic Data Generation in Speech Dysfluency Detectioncited by 8
  3. 2025Dysfluent WFST: A Framework for Zero-Shot Speech Dysfluency Transcription and Detectioncited by 5
  4. 2025Whisper Has an Internal Word Alignercited by 5
  5. 2025Seamless Dysfluent Speech Text Alignment for Disordered Speech Analysiscited by 4
  6. 2026HuPER: A Human-Inspired Framework for Phonetic Perceptioncited by 3
  7. 2025K-Function: Joint Pronunciation Transcription and Feedback for Evaluating Kids Language Functioncited by 1
  8. 2025Enabling Conversational Behavior Reasoning Capabilities in Full-Duplex Speechcited by 1
  9. 2025Revisiting Rule-Based Stuttering Detection: A Comprehensive Analysis of Interpretable Models for Clinical Applicationscited by 0
  10. 2025A Comparative Study of Controllability, Explainability, and Performance in Dysfluency Detection Modelscited by 0
  11. 2025Deploying UDM Series in Real-Life Stuttered Speech Applications: A Clinical Evaluation Frameworkcited by 0
  12. 2025SpikeVox: Towards Energy-Efficient Speech Therapy Framework with Spike-driven Generative Language Modelscited by 0
1 more
  1. 2026HASS: Hierarchical Simulation of Logopenic Aphasic Speech for Scalable PPA Detectioncited by 0