# An Effective End-to-End Modeling Approach for Mispronunciation Detection

arXiv: https://arxiv.org/abs/2005.08440 · HTML: https://mlsanity.com/paper/2005.08440/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 May 2020.

## Built on by (most cited first, 8 of 8)

- [Prompt Engineering for Healthcare: Methodologies and Applications](https://mlsanity.com/paper/2304.14670/) (2023)
- [Automatic Pronunciation Assessment using Self-Supervised Speech Representation Learning](https://mlsanity.com/paper/2204.03863/) (2022)
- [Improving Mispronunciation Detection with Wav2vec2-based Momentum Pseudo-Labeling for Accentedness and Intelligibility Assessment](https://mlsanity.com/paper/2203.15937/) (2022)
- [Phonological Level wav2vec2-based Mispronunciation Detection and Diagnosis Method](https://mlsanity.com/paper/2311.07037/) (2023)
- [Improving End-To-End Modeling for Mispronunciation Detection with Effective Augmentation Mechanisms](https://mlsanity.com/paper/2110.08731/) (2021)
- [Exploring Non-Autoregressive End-To-End Neural Modeling For English Mispronunciation Detection And Diagnosis](https://mlsanity.com/paper/2111.00844/) (2021)
- [Unseen Speaker and Language Adaptation for Lightweight Text-To-Speech with Adapters](https://mlsanity.com/paper/2508.18006/) (2025)
- [Enhancing Quranic Learning: A Multimodal Deep Learning Approach for Arabic Phoneme Recognition](https://mlsanity.com/paper/2511.17477/) (2025)

## Abstract (arXiv)

Recently, end-to-end (E2E) automatic speech recognition (ASR) systems have garnered tremendous attention because of their great success and unified modeling paradigms in comparison to conventional hybrid DNN-HMM ASR systems. Despite the widespread adoption of E2E modeling frameworks on ASR, there still is a dearth of work on investigating the E2E frameworks for use in computer-assisted pronunciation learning (CAPT), particularly for Mispronunciation detection (MD). In response, we first present a novel use of hybrid CTCAttention approach to the MD task, taking advantage of the strengths of both CTC and the attention-based model meanwhile getting around the need for phone-level forced alignment. Second, we perform input augmentation with text prompt information to make the resulting E2E model more tailored for the MD task. On the other hand, we adopt two MD decision methods so as to better cooperate with the proposed framework: 1) decision-making based on a recognition confidence measure or 2) simply based on speech recognition results. A series of Mandarin MD experiments demonstrate that our approach not only simplifies the processing pipeline of existing hybrid DNN-HMM systems but also brings about systematic and substantial performance improvements. Furthermore, input augmentation with text prompts seems to hold excellent promise for the E2E-based MD approach.