mlsanityData to 14 Oct 2026

Light-weight Pronunciation Assessment via Discrete Speech Token Surprisal

arXiv 2606.19910 ↗First posted 18 Jun 2026cs.CL cs.SD eess.AScited by 0

Not on the map. Why?

Abstract (arXiv)

Training automated pronunciation assessment often relies on labeled learner errors or non-native corpora that are costly to collect. We propose a lightweight framework trained only on native speech resources, operating unsupervised or lightly calibrated with a small set of scored utterances. At inference, learner speech is discretized with an SSL encoder and a K-means codebook. A token language model trained on native sequences computes surprisal where higher surprisal indicates phonotactic deviation. We add a transcript-guided Text2DUnit--DTW module that predicts native token sequences from reference text and aligns them to acoustic tokens to derive error-sensitive features. Surprisal and alignment features are fused via simple regression. On SpeechOcean762, PCC improves from 0.60 to 0.66 with transcript guidance, near supervised baselines. Cross-dataset evaluation on L2-ARCTIC shows consistent gains.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 38 references that are in the map. First posted on arXiv 18 Jun 2026.

Builds on

24 of 38 references.

References · 24

  1. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  2. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  3. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  4. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 601
  5. 2021Generative Spoken Language Modeling from Raw Audiocited by 340
  6. 2021CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representationcited by 232
  7. 2021speechocean762: An Open-Source Non-native English Speech Corpus For Pronunciation Assessmentcited by 58
  8. 2022Transformer-Based Multi-Aspect Multi-Granularity Non-Native English Speaker Pronunciation Assessmentcited by 37
  9. 2022Automatic Pronunciation Assessment using Self-Supervised Speech Representation Learningcited by 29
  10. 2021A Full Text-Dependent End to End Mispronunciation Detection and Diagnosis with Easy Data Augmentation Techniquescited by 19
  11. 2022Improving Mispronunciation Detection with Wav2vec2-based Momentum Pseudo-Labeling for Accentedness and Intelligibility Assessmentcited by 10
  12. 2020Detection of Lexical Stress Errors in Non-Native (L2) English with Data Augmentation and Attentioncited by 9
12 more
  1. 2020Towards Real-time Mispronunciation Detection in Kids' Speechcited by 8
  2. 2023MultiPA: A Multi-task Speech Pronunciation Assessment Model for Open Response Scenarioscited by 8
  3. 2023Multi-View Multi-Task Representation Learning for Mispronunciation Detectioncited by 6
  4. 2025Towards Efficient and Multifaceted Computer-assisted Pronunciation Training Leveraging Hierarchical Selective State Space Model and Decoupled Cross-entropy Losscited by 6
  5. 2023Zero-Shot Automatic Pronunciation Assessmentcited by 5
  6. 2025Exploring Self-Supervised Audio Models for Generalized Anomalous Sound Detectioncited by 5
  7. 2024Children's Speech Recognition through Discrete Token Enhancementcited by 4
  8. 2022Improving Non-native Word-level Pronunciation Scoring with Phone-level Mixup Data Augmentation and Multi-source Informationcited by 3
  9. 2023L1-aware Multilingual Mispronunciation Detection Frameworkcited by 3
  10. 2022SpeechBlender: Speech Augmentation Framework for Mispronunciation Data Generationcited by 2
  11. 2024Zero-Shot Text-to-Speech as Golden Speech Generator: A Systematic Framework and its Applicability in Automatic Pronunciation Assessmentcited by 1
  12. 2025Towards few-shot isolated word reading assessmentcited by 1

Built on by

Not cited yet.