# An End-to-End Mispronunciation Detection System for L2 English Speech Leveraging Novel Anti-Phone Modeling

arXiv: https://arxiv.org/abs/2005.11950 · HTML: https://mlsanity.com/paper/2005.11950/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 May 2020.

## Built on by (most cited first, 18 of 18)

- [speechocean762: An Open-Source Non-native English Speech Corpus For Pronunciation Assessment](https://mlsanity.com/paper/2104.01378/) (2021)
- [Automatic Pronunciation Assessment using Self-Supervised Speech Representation Learning](https://mlsanity.com/paper/2204.03863/) (2022)
- [A Full Text-Dependent End to End Mispronunciation Detection and Diagnosis with Easy Data Augmentation Techniques](https://mlsanity.com/paper/2104.08428/) (2021)
- [Improving Mispronunciation Detection with Wav2vec2-based Momentum Pseudo-Labeling for Accentedness and Intelligibility Assessment](https://mlsanity.com/paper/2203.15937/) (2022)
- [Computer-assisted Pronunciation Training -- Speech synthesis is almost all you need](https://mlsanity.com/paper/2207.00774/) (2022)
- [End-to-End Word-Level Pronunciation Assessment with MASK Pre-training](https://mlsanity.com/paper/2306.02682/) (2023)
- [Improving Non-native Word-level Pronunciation Scoring with Phone-level Mixup Data Augmentation and Multi-source Information](https://mlsanity.com/paper/2203.01826/) (2022)
- [Text-Aware End-to-end Mispronunciation Detection and Diagnosis](https://mlsanity.com/paper/2206.07289/) (2022)
- [Phonological Level wav2vec2-based Mispronunciation Detection and Diagnosis Method](https://mlsanity.com/paper/2311.07037/) (2023)
- [Improving End-To-End Modeling for Mispronunciation Detection with Effective Augmentation Mechanisms](https://mlsanity.com/paper/2110.08731/) (2021)
- [Self-supervised Speech Comparison for L2 Phone, Rhythm, and Intonation Scoring](https://mlsanity.com/paper/2607.13721/) (2026)
- [Incorporating L2 Phonemes Using Articulatory Features for Robust Speech Recognition](https://mlsanity.com/paper/2306.02534/) (2023)
- [Exploring Non-Autoregressive End-To-End Neural Modeling For English Mispronunciation Detection And Diagnosis](https://mlsanity.com/paper/2111.00844/) (2021)
- [CoCA-MDD: A Coupled Cross-Attention based Framework for Streaming Mispronunciation Detection and Diagnosis](https://mlsanity.com/paper/2111.08191/) (2021)
- [Pitch-Aware RNN-T for Mandarin Chinese Mispronunciation Detection and Diagnosis](https://mlsanity.com/paper/2406.04595/) (2024)
- [Unseen Speaker and Language Adaptation for Lightweight Text-To-Speech with Adapters](https://mlsanity.com/paper/2508.18006/) (2025)
- [MuFFIN: Multifaceted Pronunciation Feedback Model with Interactive Hierarchical Neural Modeling](https://mlsanity.com/paper/2510.04956/) (2025)
- [Domain-Aware Mispronunciation Detection and Diagnosis Using Language-Specific Statistical Graphs](https://mlsanity.com/paper/2606.05569/) (2026)

## Abstract (arXiv)

Mispronunciation detection and diagnosis (MDD) is a core component of computer-assisted pronunciation training (CAPT). Most of the existing MDD approaches focus on dealing with categorical errors (viz. one canonical phone is substituted by another one, aside from those mispronunciations caused by deletions or insertions). However, accurate detection and diagnosis of non-categorial or distortion errors (viz. approximating L2 phones with L1 (first-language) phones, or erroneous pronunciations in between) still seems out of reach. In view of this, we propose to conduct MDD with a novel end- to-end automatic speech recognition (E2E-based ASR) approach. In particular, we expand the original L2 phone set with their corresponding anti-phone set, making the E2E-based MDD approach have a better capability to take in both categorical and non-categorial mispronunciations, aiming to provide better mispronunciation detection and diagnosis feedback. Furthermore, a novel transfer-learning paradigm is devised to obtain the initial model estimate of the E2E-based MDD system without resource to any phonological rules. Extensive sets of experimental results on the L2-ARCTIC dataset show that our best system can outperform the existing E2E baseline system and pronunciation scoring based method (GOP) in terms of the F1-score, by 11.05% and 27.71%, respectively.