# Hierarchical Pronunciation Assessment with Multi-Aspect Attention

arXiv: https://arxiv.org/abs/2211.08102 · HTML: https://mlsanity.com/paper/2211.08102/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 22 references that are in the map.
- First posted on arXiv 15 Nov 2022.

## Builds on (most cited first, 5 of 5)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [speechocean762: An Open-Source Non-native English Speech Corpus For Pronunciation Assessment](https://mlsanity.com/paper/2104.01378/) (2021)
- [Transformer-Based Multi-Aspect Multi-Granularity Non-Native English Speaker Pronunciation Assessment](https://mlsanity.com/paper/2205.03432/) (2022)
- [Context-aware Goodness of Pronunciation for Computer-Assisted Pronunciation Training](https://mlsanity.com/paper/2008.08647/) (2020)
- [3M: An Effective Multi-view, Multi-granularity, and Multi-aspect Modeling Approach to English Pronunciation Assessment](https://mlsanity.com/paper/2208.09110/) (2022)

## Built on by (most cited first, 13 of 13)

- [MultiPA: A Multi-task Speech Pronunciation Assessment Model for Open Response Scenarios](https://mlsanity.com/paper/2308.12490/) (2023)
- [Automatic Pronunciation Assessment -- A Review](https://mlsanity.com/paper/2310.13974/) (2023)
- [Towards Efficient and Multifaceted Computer-assisted Pronunciation Training Leveraging Hierarchical Selective State Space Model and Decoupled Cross-entropy Loss](https://mlsanity.com/paper/2502.07575/) (2025)
- [Score-balanced Loss for Multi-aspect Pronunciation Assessment](https://mlsanity.com/paper/2305.16664/) (2023)
- [Pronunciation Assessment with Multi-modal Large Language Models](https://mlsanity.com/paper/2407.09209/) (2024)
- [Preserving Phonemic Distinctions for Ordinal Regression: A Novel Loss Function for Automatic Pronunciation Assessment](https://mlsanity.com/paper/2310.01839/) (2023)
- [Acoustic Feature Mixup for Balanced Multi-aspect Pronunciation Assessment](https://mlsanity.com/paper/2406.15723/) (2024)
- [Zero-Shot Text-to-Speech as Golden Speech Generator: A Systematic Framework and its Applicability in Automatic Pronunciation Assessment](https://mlsanity.com/paper/2409.07151/) (2024)
- [Fine-Tuning Large Multimodal Models for Automatic Pronunciation Assessment](https://mlsanity.com/paper/2509.15701/) (2025)
- [Multi-task Pretraining for Enhancing Interpretable L2 Pronunciation Assessment](https://mlsanity.com/paper/2509.16876/) (2025)
- [MuFFIN: Multifaceted Pronunciation Feedback Model with Interactive Hierarchical Neural Modeling](https://mlsanity.com/paper/2510.04956/) (2025)
- [When Speech Meets Lips: Interpretable Audio-Visual Synchronization for L2 Pronunciation Assessment](https://mlsanity.com/paper/2609.06788/) (2026)
- [OpenPA: Forced-alignment-free Multi-granular Pronunciation Assessment with Mixed-transcript Adaptation](https://mlsanity.com/paper/2610.13826/) (2026)

## Abstract (arXiv)

Automatic pronunciation assessment is a major component of a computer-assisted pronunciation training system. To provide in-depth feedback, scoring pronunciation at various levels of granularity such as phoneme, word, and utterance, with diverse aspects such as accuracy, fluency, and completeness, is essential. However, existing multi-aspect multi-granularity methods simultaneously predict all aspects at all granularity levels; therefore, they have difficulty in capturing the linguistic hierarchy of phoneme, word, and utterance. This limitation further leads to neglecting intimate cross-aspect relations at the same linguistic unit. In this paper, we propose a Hierarchical Pronunciation Assessment with Multi-aspect Attention (HiPAMA) model, which hierarchically represents the granularity levels to directly capture their linguistic structures and introduces multi-aspect attention that reflects associations across aspects at the same level to create more connotative representations. By obtaining relational information from both the granularity- and aspect-side, HiPAMA can take full advantage of multi-task learning. Remarkable improvements in the experimental results on the speachocean762 datasets demonstrate the robustness of HiPAMA, particularly in the difficult-to-assess aspects.