# MultiPA: A Multi-task Speech Pronunciation Assessment Model for Open Response Scenarios

arXiv: https://arxiv.org/abs/2308.12490 · HTML: https://mlsanity.com/paper/2308.12490/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 33 references that are in the map.
- First posted on arXiv 24 Aug 2023.

## Builds on (most cited first, 18 of 18)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [speechocean762: An Open-Source Non-native English Speech Corpus For Pronunciation Assessment](https://mlsanity.com/paper/2104.01378/) (2021)
- [Transformer-Based Multi-Aspect Multi-Granularity Non-Native English Speaker Pronunciation Assessment](https://mlsanity.com/paper/2205.03432/) (2022)
- [Automatic Pronunciation Assessment using Self-Supervised Speech Representation Learning](https://mlsanity.com/paper/2204.03863/) (2022)
- [Phone-to-audio alignment without text: A Semi-supervised Approach](https://mlsanity.com/paper/2110.03876/) (2021)
- [3M: An Effective Multi-view, Multi-granularity, and Multi-aspect Modeling Approach to English Pronunciation Assessment](https://mlsanity.com/paper/2208.09110/) (2022)
- [Hierarchical Pronunciation Assessment with Multi-Aspect Attention](https://mlsanity.com/paper/2211.08102/) (2022)
- [A Hierarchical Context-aware Modeling Approach for Multi-aspect and Multi-granular Pronunciation Assessment](https://mlsanity.com/paper/2305.18146/) (2023)
- [InQSS: a speech intelligibility and quality assessment model using a multi-task learning network](https://mlsanity.com/paper/2111.02585/) (2021)
- [MTI-Net: A Multi-Target Speech Intelligibility Prediction Model](https://mlsanity.com/paper/2204.03310/) (2022)
- [An ASR-free Fluency Scoring Approach with Self-Supervised Learning](https://mlsanity.com/paper/2302.09928/) (2023)
- [Zero-Shot Automatic Pronunciation Assessment](https://mlsanity.com/paper/2305.19563/) (2023)
- [Score-balanced Loss for Multi-aspect Pronunciation Assessment](https://mlsanity.com/paper/2305.16664/) (2023)
- [End-to-End Word-Level Pronunciation Assessment with MASK Pre-training](https://mlsanity.com/paper/2306.02682/) (2023)
- [Phonetic and Prosody-aware Self-supervised Learning Approach for Non-native Fluency Scoring](https://mlsanity.com/paper/2305.11438/) (2023)
- [Leveraging phone-level linguistic-acoustic similarity for utterance-level pronunciation scoring](https://mlsanity.com/paper/2302.10444/) (2023)

## Built on by (most cited first, 8 of 8)

- [HuPER: A Human-Inspired Framework for Phonetic Perception](https://mlsanity.com/paper/2602.01634/) (2026)
- [Automated evaluation of children's speech fluency for low-resource languages](https://mlsanity.com/paper/2505.19671/) (2025)
- [Scaling Human and G2P Supervision for Robust Phonetic Transcription](https://mlsanity.com/paper/2606.16019/) (2026)
- [Multi-granularity Interactive Attention Framework for Residual Hierarchical Pronunciation Assessment](https://mlsanity.com/paper/2601.01745/) (2026)
- [Light-weight Pronunciation Assessment via Discrete Speech Token Surprisal](https://mlsanity.com/paper/2606.19910/) (2026)
- [When Speech Meets Lips: Interpretable Audio-Visual Synchronization for L2 Pronunciation Assessment](https://mlsanity.com/paper/2609.06788/) (2026)
- [Deep Learning Techniques for Phoneme Recognition in Italian Children' s Speech](https://mlsanity.com/paper/2609.33060/) (2026)
- [OpenPA: Forced-alignment-free Multi-granular Pronunciation Assessment with Mixed-transcript Adaptation](https://mlsanity.com/paper/2610.13826/) (2026)

## Abstract (arXiv)

Pronunciation assessment models designed for open response scenarios enable users to practice language skills in a manner similar to real-life communication. However, previous open-response pronunciation assessment models have predominantly focused on a single pronunciation task, such as sentence-level accuracy, rather than offering a comprehensive assessment in various aspects. We propose MultiPA, a Multitask Pronunciation Assessment model that provides sentence-level accuracy, fluency, prosody, and word-level accuracy assessment for open responses. We examined the correlation between different pronunciation tasks and showed the benefits of multi-task learning. Our model reached the state-of-the-art performance on existing in-domain data sets and effectively generalized to an out-of-domain dataset that we newly collected. The experimental results demonstrate the practical utility of our model in real-world applications.