# Self-supervised Speech Comparison for L2 Phone, Rhythm, and Intonation Scoring

arXiv: https://arxiv.org/abs/2607.13721 · HTML: https://mlsanity.com/paper/2607.13721/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 41 references that are in the map.
- First posted on arXiv 15 Jul 2026.

## Builds on (most cited first, 15 of 15)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [Voice Conversion With Just Nearest Neighbors](https://mlsanity.com/paper/2305.18975/) (2023)
- [Transformer-Based Multi-Aspect Multi-Granularity Non-Native English Speaker Pronunciation Assessment](https://mlsanity.com/paper/2205.03432/) (2022)
- [Context-aware Goodness of Pronunciation for Computer-Assisted Pronunciation Training](https://mlsanity.com/paper/2008.08647/) (2020)
- [An End-to-End Mispronunciation Detection System for L2 English Speech Leveraging Novel Anti-Phone Modeling](https://mlsanity.com/paper/2005.11950/) (2020)
- [A layer-wise analysis of Mandarin and English suprasegmentals in SSL speech models](https://mlsanity.com/paper/2408.13678/) (2024)
- [Neural Representations for Modeling Variation in Speech](https://mlsanity.com/paper/2011.12649/) (2020)
- [High-Fidelity Neural Phonetic Posteriorgrams](https://mlsanity.com/paper/2402.17735/) (2024)
- [LinearVC: Linear transformations of self-supervised features through the lens of voice conversion](https://mlsanity.com/paper/2506.01510/) (2025)
- [Exploring Disentangled Neural Speech Codecs from Self-Supervised Representations](https://mlsanity.com/paper/2508.08399/) (2025)
- [Prosodic ABX: A Language-Agnostic Method for Measuring Prosodic Contrast in Speech Representations](https://mlsanity.com/paper/2604.02102/) (2026)
- [QR-VC: Leveraging Quantization Residuals for Linear Disentanglement in Zero-Shot Voice Conversion](https://mlsanity.com/paper/2411.16147/) (2024)

## Built on by (most cited first, 2 of 2)

- [A Native-Reference Coordinate Geometry for L2 Pronunciation Deviation Using Self-Supervised Speech Models](https://mlsanity.com/paper/2609.28060/) (2026)
- [A Native-Reference Phone-Class Geometry for Second-Language Pronunciation Analysis](https://mlsanity.com/paper/2609.30075/) (2026)

## Abstract (arXiv)

L2 speech assessment has traditionally focused on phonetic assessment, leaving the scoring of suprasegmental features such as rhythm and intonation underexplored. Moreover, assessment methods often require training with labeled L2 speech data, making them difficult to apply in low-resource settings. We investigate whether DTW over self-supervised WavLM representations can provide a text-free framework for assessing phonetic accuracy, rhythm, and intonation in English and Japanese L2 speech. Results show that a basic DTW-based approach that compares learner speech to native templates exceeds human agreement on holistic and sentence-level phonetic scoring. For rhythm, we introduce methods that measure the degree of warping in the DTW alignment path; our best method approaches human-level performance. For intonation, we combine DTW distance over prosodic residuals with pitch and intensity features, but performance remains more modest on some tasks. Our results point to self-supervised representations as a promising, text-free basis for multi-aspect pronunciation assessment.