# An Approach to Mispronunciation Detection and Diagnosis with Acoustic, Phonetic and Linguistic (APL) Embeddings

arXiv: https://arxiv.org/abs/2110.07274 · HTML: https://mlsanity.com/paper/2110.07274/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 20 references that are in the map.
- First posted on arXiv 14 Oct 2021.

## Builds on (most cited first, 3 of 3)

- [Any-to-Many Voice Conversion with Location-Relative Sequence-to-Sequence Modeling](https://mlsanity.com/paper/2009.02725/) (2020)
- [Context-aware Goodness of Pronunciation for Computer-Assisted Pronunciation Training](https://mlsanity.com/paper/2008.08647/) (2020)
- [A Full Text-Dependent End to End Mispronunciation Detection and Diagnosis with Easy Data Augmentation Techniques](https://mlsanity.com/paper/2104.08428/) (2021)

## Built on by (most cited first, 8 of 8)

- [Automatic Pronunciation Assessment -- A Review](https://mlsanity.com/paper/2310.13974/) (2023)
- [Phonological Level wav2vec2-based Mispronunciation Detection and Diagnosis Method](https://mlsanity.com/paper/2311.07037/) (2023)
- [Whisper based Cross-Lingual Phoneme Recognition between Vietnamese and English](https://mlsanity.com/paper/2508.19270/) (2025)
- [Fine-Tuning Large Multimodal Models for Automatic Pronunciation Assessment](https://mlsanity.com/paper/2509.15701/) (2025)
- [DTW-SiameseNet: Dynamic Time Warped Siamese Network for Mispronunciation Detection and Correction](https://mlsanity.com/paper/2303.00171/) (2023)
- [JCAPT: A Joint Modeling Approach for CAPT](https://mlsanity.com/paper/2506.19315/) (2025)
- [Mispronunciation Detection and Diagnosis Without Model Training: A Retrieval-Based Approach](https://mlsanity.com/paper/2511.20107/) (2025)
- [Domain-Aware Mispronunciation Detection and Diagnosis Using Language-Specific Statistical Graphs](https://mlsanity.com/paper/2606.05569/) (2026)

## Abstract (arXiv)

Many mispronunciation detection and diagnosis (MD&D) research approaches try to exploit both the acoustic and linguistic features as input. Yet the improvement of the performance is limited, partially due to the shortage of large amount annotated training data at the phoneme level. Phonetic embeddings, extracted from ASR models trained with huge amount of word level annotations, can serve as a good representation of the content of input speech, in a noise-robust and speaker-independent manner. These embeddings, when used as implicit phonetic supplementary information, can alleviate the data shortage of explicit phoneme annotations. We propose to utilize Acoustic, Phonetic and Linguistic (APL) embedding features jointly for building a more powerful MD&D system. Experimental results obtained on the L2-ARCTIC database show the proposed approach outperforms the baseline by 9.93%, 10.13% and 6.17% on the detection accuracy, diagnosis error rate and the F-measure, respectively.