# Predicting within and across language phoneme recognition performance of self-supervised learning speech pre-trained models

arXiv: https://arxiv.org/abs/2206.12489 · HTML: https://mlsanity.com/paper/2206.12489/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jun 2022.

## Built on by (most cited first, 7 of 7)

- [Comparative layer-wise analysis of self-supervised speech models](https://mlsanity.com/paper/2211.03929/) (2022)
- [What Do Self-Supervised Speech Models Know About Words?](https://mlsanity.com/paper/2307.00162/) (2023)
- [Self-Supervised Video Transformers for Isolated Sign Language Recognition](https://mlsanity.com/paper/2309.02450/) (2023)
- [Uncertainty as a Predictor: Leveraging Self-Supervised Learning for Zero-Shot MOS Prediction](https://mlsanity.com/paper/2312.15616/) (2023)
- [Counterfactual Activation Editing for Post-hoc Prosody and Mispronunciation Correction in TTS Models](https://mlsanity.com/paper/2506.00832/) (2025)
- [Emergence of Phonemic, Syntactic, and Semantic Representations in Artificial Neural Networks](https://mlsanity.com/paper/2601.18617/) (2026)
- [How Far Do SSL Speech Models Listen for Tone? Temporal Focus of Tone Representation under Low-resource Transfer](https://mlsanity.com/paper/2511.12285/) (2025)

## Abstract (arXiv)

In this work, we analyzed and compared speech representations extracted from different frozen self-supervised learning (SSL) speech pre-trained models on their ability to capture articulatory features (AF) information and their subsequent prediction of phone recognition performance for within and across language scenarios. Specifically, we compared CPC, wav2vec 2.0, and HuBert. First, frame-level AF probing tasks were implemented. Subsequently, phone-level end-to-end ASR systems for phoneme recognition tasks were implemented, and the performance on the frame-level AF probing task and the phone accuracy were correlated. Compared to the conventional speech representation MFCC, all SSL pre-trained speech representations captured more AF information, and achieved better phoneme recognition performance within and across languages, with HuBert performing best. The frame-level AF probing task is a good predictor of phoneme recognition performance, showing the importance of capturing AF information in the speech representations. Compared with MFCC, in the within-language scenario, the performance of these SSL speech pre-trained models on AF probing tasks achieved a maximum relative increase of 34.4%, and it resulted in the lowest PER of 10.2%. In the cross-language scenario, the maximum relative increase of 26.7% also resulted in the lowest PER of 23.0%.