mlsanityData to 14 Oct 2026

JCAPT: A Joint Modeling Approach for CAPT

arXiv 2506.19315 ↗First posted 24 Jun 2025cs.CL cs.AI eess.AScited by 0

On the map: State-space sequence models (S4, Mamba) → Speech and audio SSMs

Abstract (arXiv)

Effective pronunciation feedback is critical in second language (L2) learning, for which computer-assisted pronunciation training (CAPT) systems often encompass two key tasks: automatic pronunciation assessment (APA) and mispronunciation detection and diagnosis (MDD). Recent work has shown that joint modeling of these two tasks can yield mutual benefits. Our unified framework leverages Mamba, a selective state space model (SSM), while integrating phonological features and think token strategies to jointly enhance interpretability and fine-grained temporal reasoning in APA and MDD. To our knowledge, this is the first study to combine phonological attribution, SSM-based modeling, and prompting in CAPT. A series of experiments conducted on the speechocean762 benchmark demonstrate that our model consistently outperforms prior methods, particularly on the MDD task.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Speech and audio SSMs, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 13 of its 26 references that are in the map. First posted on arXiv 24 Jun 2025.

Builds on

13 of 26 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Dual-path Mamba: Short and Long-term Bidirectional Selective Structured State Space Models for Speech Separation27 Mar

Other references · 12

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  4. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  5. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  6. 2021speechocean762: An Open-Source Non-native English Speech Corpus For Pronunciation Assessmentcited by 58
  7. 2022Transformer-Based Multi-Aspect Multi-Granularity Non-Native English Speaker Pronunciation Assessmentcited by 37
  8. 2020Context-aware Goodness of Pronunciation for Computer-Assisted Pronunciation Trainingcited by 26
  9. 2024MambaPEFT: Exploring Parameter-Efficient Fine-Tuning for Mambacited by 11
  10. 2021An Approach to Mispronunciation Detection and Diagnosis with Acoustic, Phonetic and Linguistic (APL) Embeddingscited by 8
  11. 2025Towards Efficient and Multifaceted Computer-assisted Pronunciation Training Leveraging Hierarchical Selective State Space Model and Decoupled Cross-entropy Losscited by 6
  12. 2023Preserving Phonemic Distinctions for Ordinal Regression: A Novel Loss Function for Automatic Pronunciation Assessmentcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Fine-Tuning Large Multimodal Models for Automatic Pronunciation Assessment
  2. 2025Exploring the Potential of Large Multimodal Models as Effective Alternatives for Pronunciation Assessment
  3. 2024Pronunciation Assessment with Multi-modal Large Language Models
  4. 2025PAC: Pronunciation-Aware Contextualized Large Language Model-based Automatic Speech Recognition
  5. 2025SAM: A Mamba-2 State-Space Audio-Language Model
  6. 2024DeSTA2: Developing Instruction-Following Speech Language Model Without Speech Instruction-Tuning Data
  7. 2026Prior over Evidence: Stereotype-Driven Diagnosis in LLM-Based L2 Pronunciation Feedback
  8. 2025English Pronunciation Evaluation without Complex Joint Training: LoRA Fine-tuned Speech Multimodal LLM