# Light-weight Pronunciation Assessment via Discrete Speech Token Surprisal

arXiv: https://arxiv.org/abs/2606.19910 · HTML: https://mlsanity.com/paper/2606.19910/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 38 references that are in the map.
- First posted on arXiv 18 Jun 2026.

## Builds on (most cited first, 24 of 24)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale](https://mlsanity.com/paper/2111.09296/) (2021)
- [Generative Spoken Language Modeling from Raw Audio](https://mlsanity.com/paper/2102.01192/) (2021)
- [CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representation](https://mlsanity.com/paper/2103.06874/) (2021)
- [speechocean762: An Open-Source Non-native English Speech Corpus For Pronunciation Assessment](https://mlsanity.com/paper/2104.01378/) (2021)
- [Transformer-Based Multi-Aspect Multi-Granularity Non-Native English Speaker Pronunciation Assessment](https://mlsanity.com/paper/2205.03432/) (2022)
- [Automatic Pronunciation Assessment using Self-Supervised Speech Representation Learning](https://mlsanity.com/paper/2204.03863/) (2022)
- [A Full Text-Dependent End to End Mispronunciation Detection and Diagnosis with Easy Data Augmentation Techniques](https://mlsanity.com/paper/2104.08428/) (2021)
- [Improving Mispronunciation Detection with Wav2vec2-based Momentum Pseudo-Labeling for Accentedness and Intelligibility Assessment](https://mlsanity.com/paper/2203.15937/) (2022)
- [Detection of Lexical Stress Errors in Non-Native (L2) English with Data Augmentation and Attention](https://mlsanity.com/paper/2012.14788/) (2020)
- [Towards Real-time Mispronunciation Detection in Kids' Speech](https://mlsanity.com/paper/2003.01765/) (2020)
- [MultiPA: A Multi-task Speech Pronunciation Assessment Model for Open Response Scenarios](https://mlsanity.com/paper/2308.12490/) (2023)
- [Multi-View Multi-Task Representation Learning for Mispronunciation Detection](https://mlsanity.com/paper/2306.01845/) (2023)
- [Towards Efficient and Multifaceted Computer-assisted Pronunciation Training Leveraging Hierarchical Selective State Space Model and Decoupled Cross-entropy Loss](https://mlsanity.com/paper/2502.07575/) (2025)
- [Zero-Shot Automatic Pronunciation Assessment](https://mlsanity.com/paper/2305.19563/) (2023)
- [Exploring Self-Supervised Audio Models for Generalized Anomalous Sound Detection](https://mlsanity.com/paper/2508.12230/) (2025)
- [Children's Speech Recognition through Discrete Token Enhancement](https://mlsanity.com/paper/2406.13431/) (2024)
- [Improving Non-native Word-level Pronunciation Scoring with Phone-level Mixup Data Augmentation and Multi-source Information](https://mlsanity.com/paper/2203.01826/) (2022)
- [L1-aware Multilingual Mispronunciation Detection Framework](https://mlsanity.com/paper/2309.07719/) (2023)
- [SpeechBlender: Speech Augmentation Framework for Mispronunciation Data Generation](https://mlsanity.com/paper/2211.00923/) (2022)
- [Zero-Shot Text-to-Speech as Golden Speech Generator: A Systematic Framework and its Applicability in Automatic Pronunciation Assessment](https://mlsanity.com/paper/2409.07151/) (2024)
- [Towards few-shot isolated word reading assessment](https://mlsanity.com/paper/2507.12217/) (2025)

## Abstract (arXiv)

Training automated pronunciation assessment often relies on labeled learner errors or non-native corpora that are costly to collect. We propose a lightweight framework trained only on native speech resources, operating unsupervised or lightly calibrated with a small set of scored utterances. At inference, learner speech is discretized with an SSL encoder and a K-means codebook. A token language model trained on native sequences computes surprisal where higher surprisal indicates phonotactic deviation. We add a transcript-guided Text2DUnit--DTW module that predicts native token sequences from reference text and aligns them to acoustic tokens to derive error-sensitive features. Surprisal and alignment features are fused via simple regression. On SpeechOcean762, PCC improves from 0.60 to 0.66 with transcript guidance, near supervised baselines. Cross-dataset evaluation on L2-ARCTIC shows consistent gains.