# A layer-wise analysis of Mandarin and English suprasegmentals in SSL speech models arXiv: https://arxiv.org/abs/2408.13678 ยท HTML: https://mlsanity.com/paper/2408.13678/ ## Key facts - Cited by 18 papers in the map (data through 14 Oct 2026). - Builds on 16 of its 25 references that are in the map. - First posted on arXiv 24 Aug 2024. ## Builds on (most cited first, 16 of 16) - [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020) - [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021) - [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021) - [SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives](https://mlsanity.com/paper/1407.0202/) (2014) - [Layer-wise Analysis of a Self-supervised Speech Representation Model](https://mlsanity.com/paper/2107.04734/) (2021) - [Comparative layer-wise analysis of self-supervised speech models](https://mlsanity.com/paper/2211.03929/) (2022) - [What Do Self-Supervised Speech Models Know About Words?](https://mlsanity.com/paper/2307.00162/) (2023) - [Conditional probing: measuring usable information beyond a baseline](https://mlsanity.com/paper/2109.09234/) (2021) - [What all do audio transformer models hear? Probing Acoustic Representations for Language Delivery and its Structure](https://mlsanity.com/paper/2101.00387/) (2021) - [On the Utility of Self-supervised Models for Prosody-related Tasks](https://mlsanity.com/paper/2210.07185/) (2022) - [Probing self-supervised speech models for phonetic and phonemic information: a case study in aspiration](https://mlsanity.com/paper/2306.06232/) (2023) - [Exploration of A Self-Supervised Speech Model: A Study on Emotional Corpora](https://mlsanity.com/paper/2210.02595/) (2022) - [Probing Acoustic Representations for Phonetic Properties](https://mlsanity.com/paper/2010.13007/) (2020) - [What Can an Accent Identifier Learn? Probing Phonetic and Prosodic Information in a Wav2vec2-based Accent Identification Model](https://mlsanity.com/paper/2306.06524/) (2023) - [A context-aware knowledge transferring strategy for CTC-based ASR](https://mlsanity.com/paper/2210.06244/) (2022) - [Automatic recognition of suprasegmentals in speech](https://mlsanity.com/paper/2108.01122/) (2021) ## Built on by (most cited first, 18 of 18) - [Segmentation-Variant Codebooks for Preservation of Paralinguistic and Prosodic Information](https://mlsanity.com/paper/2505.15667/) (2025) - [What do self-supervised speech models know about Dutch? Analyzing advantages of language-specific pre-training](https://mlsanity.com/paper/2506.00981/) (2025) - [Analyzing the relationships between pretraining language, phonetic, tonal, and speaker information in self-supervised speech models](https://mlsanity.com/paper/2506.10855/) (2025) - [Prosodic ABX: A Language-Agnostic Method for Measuring Prosodic Contrast in Speech Representations](https://mlsanity.com/paper/2604.02102/) (2026) - [Hierarchical Self-Supervised Representation Learning for Depression Detection from Speech](https://mlsanity.com/paper/2510.08593/) (2025) - [Self-supervised Speech Comparison for L2 Phone, Rhythm, and Intonation Scoring](https://mlsanity.com/paper/2607.13721/) (2026) - [Representation of perceived prosodic similarity of conversational feedback](https://mlsanity.com/paper/2505.13268/) (2025) - [Tone recognition in low-resource languages of North-East India: peeling the layers of SSL-based speech models](https://mlsanity.com/paper/2506.03606/) (2025) - [LTA-L2S: Lexical Tone-Aware Lip-to-Speech Synthesis for Mandarin with Cross-Lingual Transfer Learning](https://mlsanity.com/paper/2509.25670/) (2025) - [How Far Do SSL Speech Models Listen for Tone? Temporal Focus of Tone Representation under Low-resource Transfer](https://mlsanity.com/paper/2511.12285/) (2025) - [SITA: Learning Speaker-Invariant and Tone-Aware Speech Representations for Low-Resource Tonal Languages](https://mlsanity.com/paper/2601.09050/) (2026) - [Tracking the emergence of linguistic structure in self-supervised models learning from speech](https://mlsanity.com/paper/2604.02043/) (2026) - [Phoneme- and Word-Level Metrics Using Self-Supervised Speech Representations for Forced Alignment Evaluation](https://mlsanity.com/paper/2608.28508/) (2026) - [Emphasis Sensitivity in Speech Representations](https://mlsanity.com/paper/2508.11566/) (2025) - [HarmoniFuse: A Component-Selective and Prompt-Adaptive Framework for Multi-Task Speech Language Modeling](https://mlsanity.com/paper/2509.18570/) (2025) - [Beyond Decodability: Reconstructing Language Model Representations with an Encoding Probe](https://mlsanity.com/paper/2605.00607/) (2026) - [Perceptual compensation for tonal context in self-supervised speech models](https://mlsanity.com/paper/2606.17835/) (2026) - [Layer-wise Probing of wav2vec 2.0 and Whisper for Consonant Cluster Reduction in African American English](https://mlsanity.com/paper/2606.23948/) (2026) ## Abstract (arXiv) This study asks how self-supervised speech models represent suprasegmental categories like Mandarin lexical tone, English lexical stress, and English phrasal accents. Through a series of probing tasks, we make layer-wise comparisons of English and Mandarin 12 layer monolingual models. Our findings suggest that 1) English and Mandarin wav2vec 2.0 models learn contextual representations of abstract suprasegmental categories which are strongest in the middle third of the network. 2) Models are better at representing features that exist in the language of their training data, and this difference is driven by enriched context in transformer blocks, not local acoustic representation. 3) Fine-tuned wav2vec 2.0 improves performance in later layers compared to pre-trained models mainly for lexically contrastive features like tone and stress, 4) HuBERT and WavLM learn similar representations to wav2vec 2.0, differing mainly in later layer performance. Our results extend previous understanding of how models represent suprasegmentals and offer new insights into the language-specificity and contextual nature of these representations.