mlsanityData to 7 Oct 2026

Leveraging Allophony in Self-Supervised Speech Models for Atypical Pronunciation Assessment

arXiv 2502.07029 ↗First posted 10 Feb 2025cs.CL cs.AI cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

Allophony refers to the variation in the phonetic realization of a phoneme based on its phonetic environment. Modeling allophones is crucial for atypical pronunciation assessment, which involves distinguishing atypical from typical pronunciations. However, recent phoneme classifier-based approaches often simplify this by treating various realizations as a single phoneme, bypassing the complexity of modeling allophonic variation. Motivated by the acoustic modeling capabilities of frozen self-supervised speech model (S3M) features, we propose MixGoP, a novel approach that leverages Gaussian mixture models to model phoneme distributions with multiple subclusters. Our experiments show that MixGoP achieves state-of-the-art performance across four out of five datasets, including dysarthric and non-native speech. Our analysis further suggests that S3M features capture allophonic variation more effectively than MFCCs and Mel spectrograms, highlighting the benefits of integrating MixGoP with S3M features.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 30 of its 72 references that are in the map. First posted on arXiv 10 Feb 2025.

Builds on

30 of 72 references.

References · 30

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  3. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  4. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  5. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  6. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  7. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 652
  8. 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 596
  9. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
  10. 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 293
  11. 2020Fast Differentiable Sorting and Rankingcited by 196
  12. 2022Comparative layer-wise analysis of self-supervised speech modelscited by 155
18 more
  1. 2024Self-Supervised Speech Representations are More Phonetic than Semanticcited by 70
  2. 2023What Do Self-Supervised Speech Models Know About Words?cited by 62
  3. 2021speechocean762: An Open-Source Non-native English Speech Corpus For Pronunciation Assessmentcited by 57
  4. 2023Analysing Discrete Self Supervised Speech Representation for Spoken Language Modelingcited by 47
  5. 2023Exploring Speech Recognition, Translation, and Understanding with Discrete Speech Units: A Comparative Studycited by 42
  6. 2022Transformer-Based Multi-Aspect Multi-Granularity Non-Native English Speaker Pronunciation Assessmentcited by 36
  7. 2023Probing self-supervised speech models for phonetic and phonemic information: a case study in aspirationcited by 35
  8. 2020Context-aware Goodness of Pronunciation for Computer-Assisted Pronunciation Trainingcited by 26
  9. 2022SUPERB @ SLT 2022: Challenge on Generalization and Efficiency of Self-Supervised Speech Representation Learningcited by 26
  10. 20223M: An Effective Multi-view, Multi-granularity, and Multi-aspect Modeling Approach to English Pronunciation Assessmentcited by 21
  11. 2023Self-supervised Predictive Coding Models Encode Speaker and Phonetic Information in Orthogonal Subspacescited by 19
  12. 2022Opening the Black Box of wav2vec Feature Encodercited by 18
  13. 2023An Information-Theoretic Analysis of Self-supervised Discrete Representations of Speechcited by 15
  14. 2023Understanding Probe Behaviors through Variational Bounds of Mutual Informationcited by 11
  15. 2023Speech Intelligibility Assessment of Dysarthric Speech by using Goodness of Pronunciation with Uncertainty Quantificationcited by 10
  16. 2021Mispronunciation Detection in Non-native (L2) English with Uncertainty Modelingcited by 5
  17. 2023Phonological Level wav2vec2-based Mispronunciation Detection and Diagnosis Methodcited by 3
  18. 2023Comparison of L2 Korean pronunciation error patterns from five L1 backgrounds by using automatic phonetic transcriptioncited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 9.

15 papers.

Sort:

  1. 2025POWSM: A Phonetic Open Whisper-Style Speech Foundation Modelcited by 14
  2. 2025Emergent morpho-phonological representations in self-supervised speech modelscited by 11
  3. 2026[b] = [d] - [t] + [p]: Self-supervised Speech Models Discover Phonological Vector Arithmeticcited by 11
  4. 2026Self-Supervised Speech Models Encode Phonetic Context via Position-dependent Orthogonal Subspacescited by 7
  5. 2025On-device Streaming Discrete Speech Unitscited by 4
  6. 2025Seamless Dysfluent Speech Text Alignment for Disordered Speech Analysiscited by 4
  7. 2026PRiSM: Benchmarking Phone Realization in Speech Modelscited by 4
  8. 2026Adapting Self-Supervised Speech Representations for Cross-lingual Dysarthria Detection in Parkinson's Diseasecited by 4
  9. 2025Towards Inclusive ASR: Investigating Voice Conversion for Dysarthric Speech Recognition in Low-Resource Languagescited by 3
  10. 2026Multilingual Phonological Feature Recognition with Self-Supervised Speech Modelscited by 2
  11. 2026A Native-Reference Coordinate Geometry for L2 Pronunciation Deviation Using Self-Supervised Speech Modelscited by 1
  12. 2025Revisiting Rule-Based Stuttering Detection: A Comprehensive Analysis of Interpretable Models for Clinical Applicationscited by 0
3 more
  1. 2026Tracking the emergence of linguistic structure in self-supervised models learning from speechcited by 0
  2. 2026Phone Segmentation and Recognition through Phonological Activation Mappingcited by 0
  3. 2026A Native-Reference Phone-Class Geometry for Second-Language Pronunciation Analysiscited by 0