# Zero-Shot Automatic Pronunciation Assessment

arXiv: https://arxiv.org/abs/2305.19563 · HTML: https://mlsanity.com/paper/2305.19563/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 25 references that are in the map.
- First posted on arXiv 31 May 2023.

## Builds on (most cited first, 12 of 12)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [The Zero Resource Speech Benchmark 2021: Metrics and baselines for unsupervised spoken language modeling](https://mlsanity.com/paper/2011.11588/) (2020)
- [A Comparison of Discrete and Soft Speech Units for Improved Voice Conversion](https://mlsanity.com/paper/2111.02392/) (2021)
- [speechocean762: An Open-Source Non-native English Speech Corpus For Pronunciation Assessment](https://mlsanity.com/paper/2104.01378/) (2021)
- [The Zero Resource Speech Challenge 2021: Spoken language modelling](https://mlsanity.com/paper/2104.14700/) (2021)
- [LRSpeech: Extremely Low-Resource Speech Synthesis and Recognition](https://mlsanity.com/paper/2008.03687/) (2020)
- [Transformer-Based Multi-Aspect Multi-Granularity Non-Native English Speaker Pronunciation Assessment](https://mlsanity.com/paper/2205.03432/) (2022)
- [Automatic Pronunciation Assessment using Self-Supervised Speech Representation Learning](https://mlsanity.com/paper/2204.03863/) (2022)
- [Analyzing Speaker Information in Self-Supervised Models to Improve Zero-Resource Speech Processing](https://mlsanity.com/paper/2108.00917/) (2021)
- [Towards Zero-shot Learning for Automatic Phonemic Transcription](https://mlsanity.com/paper/2002.11781/) (2020)

## Built on by (most cited first, 5 of 5)

- [MultiPA: A Multi-task Speech Pronunciation Assessment Model for Open Response Scenarios](https://mlsanity.com/paper/2308.12490/) (2023)
- [Segmentation-free Goodness of Pronunciation](https://mlsanity.com/paper/2507.16838/) (2025)
- [Pitch-Aware RNN-T for Mandarin Chinese Mispronunciation Detection and Diagnosis](https://mlsanity.com/paper/2406.04595/) (2024)
- [Goodness-of-pronunciation without phoneme time alignment](https://mlsanity.com/paper/2603.25150/) (2026)
- [Light-weight Pronunciation Assessment via Discrete Speech Token Surprisal](https://mlsanity.com/paper/2606.19910/) (2026)

## Abstract (arXiv)

Automatic Pronunciation Assessment (APA) is vital for computer-assisted language learning. Prior methods rely on annotated speech-text data to train Automatic Speech Recognition (ASR) models or speech-score data to train regression models. In this work, we propose a novel zero-shot APA method based on the pre-trained acoustic model, HuBERT. Our method involves encoding speech input and corrupting them via a masking module. We then employ the Transformer encoder and apply k-means clustering to obtain token sequences. Finally, a scoring module is designed to measure the number of wrongly recovered tokens. Experimental results on speechocean762 demonstrate that the proposed method achieves comparable performance to supervised regression baselines and outperforms non-regression baselines in terms of Pearson Correlation Coefficient (PCC). Additionally, we analyze how masking strategies affect the performance of APA.