# Transformer-Based Multi-Aspect Multi-Granularity Non-Native English Speaker Pronunciation Assessment

arXiv: https://arxiv.org/abs/2205.03432 · HTML: https://mlsanity.com/paper/2205.03432/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 27 references that are in the map.
- First posted on arXiv 6 May 2022.

## Builds on (most cited first, 6 of 6)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptation](https://mlsanity.com/paper/1805.04699/) (2018)
- [speechocean762: An Open-Source Non-native English Speech Corpus For Pronunciation Assessment](https://mlsanity.com/paper/2104.01378/) (2021)
- [Context-aware Goodness of Pronunciation for Computer-Assisted Pronunciation Training](https://mlsanity.com/paper/2008.08647/) (2020)
- [Investigating the role of L1 in automatic pronunciation evaluation of L2 speech](https://mlsanity.com/paper/1807.01738/) (2018)

## Built on by (most cited first, 30 of 37)

- [ZIPA: A family of efficient models for multilingual phone recognition](https://mlsanity.com/paper/2505.23170/) (2025)
- [Leveraging Allophony in Self-Supervised Speech Models for Atypical Pronunciation Assessment](https://mlsanity.com/paper/2502.07029/) (2025)
- [Hierarchical Pronunciation Assessment with Multi-Aspect Attention](https://mlsanity.com/paper/2211.08102/) (2022)
- [MultiPA: A Multi-task Speech Pronunciation Assessment Model for Open Response Scenarios](https://mlsanity.com/paper/2308.12490/) (2023)
- [Automatic Pronunciation Assessment -- A Review](https://mlsanity.com/paper/2310.13974/) (2023)
- [Towards Efficient and Multifaceted Computer-assisted Pronunciation Training Leveraging Hierarchical Selective State Space Model and Decoupled Cross-entropy Loss](https://mlsanity.com/paper/2502.07575/) (2025)
- [Zero-Shot Automatic Pronunciation Assessment](https://mlsanity.com/paper/2305.19563/) (2023)
- [Score-balanced Loss for Multi-aspect Pronunciation Assessment](https://mlsanity.com/paper/2305.16664/) (2023)
- [Pronunciation Assessment with Multi-modal Large Language Models](https://mlsanity.com/paper/2407.09209/) (2024)
- [Phonetic and Prosody-aware Self-supervised Learning Approach for Non-native Fluency Scoring](https://mlsanity.com/paper/2305.11438/) (2023)
- [Addressing Cold Start Problem for End-to-end Automatic Speech Scoring](https://mlsanity.com/paper/2306.14310/) (2023)
- [L1-aware Multilingual Mispronunciation Detection Framework](https://mlsanity.com/paper/2309.07719/) (2023)
- [Preserving Phonemic Distinctions for Ordinal Regression: A Novel Loss Function for Automatic Pronunciation Assessment](https://mlsanity.com/paper/2310.01839/) (2023)
- [Acoustic Feature Mixup for Balanced Multi-aspect Pronunciation Assessment](https://mlsanity.com/paper/2406.15723/) (2024)
- [SpeechBlender: Speech Augmentation Framework for Mispronunciation Data Generation](https://mlsanity.com/paper/2211.00923/) (2022)
- [Flowchase: a Mobile Application for Pronunciation Training](https://mlsanity.com/paper/2307.02051/) (2023)
- [Zero-Shot Speech LLMs for Multi-Aspect Evaluation of L2 Speech: Challenges and Opportunities](https://mlsanity.com/paper/2601.16230/) (2026)
- [Self-supervised Speech Comparison for L2 Phone, Rhythm, and Intonation Scoring](https://mlsanity.com/paper/2607.13721/) (2026)
- [The complementary roles of non-verbal cues for Robust Pronunciation Assessment](https://mlsanity.com/paper/2309.07739/) (2023)
- [Automated evaluation of children's speech fluency for low-resource languages](https://mlsanity.com/paper/2505.19671/) (2025)
- [Evaluating Logit-Based GOP Scores for Mispronunciation Detection](https://mlsanity.com/paper/2506.12067/) (2025)
- [Segmentation-free Goodness of Pronunciation](https://mlsanity.com/paper/2507.16838/) (2025)
- [English Pronunciation Evaluation without Complex Joint Training: LoRA Fine-tuned Speech Multimodal LLM](https://mlsanity.com/paper/2509.02915/) (2025)
- [Fine-Tuning Large Multimodal Models for Automatic Pronunciation Assessment](https://mlsanity.com/paper/2509.15701/) (2025)
- [A Finetuned SpeechLLM for Joint Multi-Granular L2 Assessment and Natural-Language Rationales](https://mlsanity.com/paper/2606.09470/) (2026)
- [Scaling Human and G2P Supervision for Robust Phonetic Transcription](https://mlsanity.com/paper/2606.16019/) (2026)
- [Mispronunciation detection using self-supervised speech representations](https://mlsanity.com/paper/2307.16324/) (2023)
- [Semi-supervised Learning For Robust Speech Evaluation](https://mlsanity.com/paper/2409.14666/) (2024)
- [JCAPT: A Joint Modeling Approach for CAPT](https://mlsanity.com/paper/2506.19315/) (2025)
- [Multi-task Pretraining for Enhancing Interpretable L2 Pronunciation Assessment](https://mlsanity.com/paper/2509.16876/) (2025)

## Abstract (arXiv)

Automatic pronunciation assessment is an important technology to help self-directed language learners. While pronunciation quality has multiple aspects including accuracy, fluency, completeness, and prosody, previous efforts typically only model one aspect (e.g., accuracy) at one granularity (e.g., at the phoneme-level). In this work, we explore modeling multi-aspect pronunciation assessment at multiple granularities. Specifically, we train a Goodness Of Pronunciation feature-based Transformer (GOPT) with multi-task learning. Experiments show that GOPT achieves the best results on speechocean762 with a public automatic speech recognition (ASR) acoustic model trained on Librispeech.