# Self-Supervised Speech Models Encode Phonetic Context via Position-dependent Orthogonal Subspaces

arXiv: https://arxiv.org/abs/2603.12642 · HTML: https://mlsanity.com/paper/2603.12642/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 51 references that are in the map.
- First posted on arXiv 13 Mar 2026.

## Builds on (most cited first, 30 of 41)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings](https://mlsanity.com/paper/1909.00512/) (2019)
- [AudioLM: a Language Modeling Approach to Audio Generation](https://mlsanity.com/paper/2209.03143/) (2022)
- [Moshi: a speech-text foundation model for real-time dialogue](https://mlsanity.com/paper/2410.00037/) (2024)
- [Layer-wise Analysis of a Self-supervised Speech Representation Model](https://mlsanity.com/paper/2107.04734/) (2021)
- [Comparative layer-wise analysis of self-supervised speech models](https://mlsanity.com/paper/2211.03929/) (2022)
- [Self-Supervised Speech Representations are More Phonetic than Semantic](https://mlsanity.com/paper/2406.08619/) (2024)
- [What Do Self-Supervised Speech Models Know About Words?](https://mlsanity.com/paper/2307.00162/) (2023)
- [Analysing Discrete Self Supervised Speech Representation for Spoken Language Modeling](https://mlsanity.com/paper/2301.00591/) (2023)
- [Exploring Speech Recognition, Translation, and Understanding with Discrete Speech Units: A Comparative Study](https://mlsanity.com/paper/2309.15800/) (2023)
- [Discrete Audio Tokens: More Than a Survey!](https://mlsanity.com/paper/2506.10274/) (2025)
- [Word Discovery in Visually Grounded, Self-Supervised Speech Models](https://mlsanity.com/paper/2203.15081/) (2022)
- [Probing self-supervised speech models for phonetic and phonemic information: a case study in aspiration](https://mlsanity.com/paper/2306.06232/) (2023)
- [Evidence of Vocal Tract Articulation in Self-Supervised Learning of Speech](https://mlsanity.com/paper/2210.11723/) (2022)
- [SyllableLM: Learning Coarse Semantic Units for Speech Language Models](https://mlsanity.com/paper/2410.04029/) (2024)
- [Self-supervised Predictive Coding Models Encode Speaker and Phonetic Information in Orthogonal Subspaces](https://mlsanity.com/paper/2305.12464/) (2023)
- [Human-like Linguistic Biases in Neural Speech Models: Phonetic Categorization and Phonotactic Constraints in Wav2Vec2.0](https://mlsanity.com/paper/2407.03005/) (2024)
- [Sylber: Syllabic Embedding Representation of Speech from Raw Audio](https://mlsanity.com/paper/2410.07168/) (2024)
- [Opening the Black Box of wav2vec Feature Encoder](https://mlsanity.com/paper/2210.15386/) (2022)
- [Wave to Syntax: Probing spoken language models for syntax](https://mlsanity.com/paper/2305.18957/) (2023)
- [Coding Speech through Vocal Tract Kinematics](https://mlsanity.com/paper/2406.12998/) (2024)
- [Self-Supervised Models of Speech Infer Universal Articulatory Kinematics](https://mlsanity.com/paper/2310.10788/) (2023)
- [An Information-Theoretic Analysis of Self-supervised Discrete Representations of Speech](https://mlsanity.com/paper/2306.02405/) (2023)
- [Leveraging Allophony in Self-Supervised Speech Models for Atypical Pronunciation Assessment](https://mlsanity.com/paper/2502.07029/) (2025)
- [[b] = [d] - [t] + [p]: Self-supervised Speech Models Discover Phonological Vector Arithmetic](https://mlsanity.com/paper/2602.18899/) (2026)
- [Understanding Probe Behaviors through Variational Bounds of Mutual Information](https://mlsanity.com/paper/2312.10019/) (2023)
- [Phonetic Segmentation of the UCLA Phonetics Lab Archive](https://mlsanity.com/paper/2403.19509/) (2024)
- [Silence is Sweeter Than Speech: Self-Supervised Model Using Silence to Store Speaker Information](https://mlsanity.com/paper/2205.03759/) (2022)

## Built on by (most cited first, 8 of 8)

- [An Empirical Recipe for Universal Phone Recognition](https://mlsanity.com/paper/2603.29042/) (2026)
- [Adapting Self-Supervised Speech Representations for Cross-lingual Dysarthria Detection in Parkinson's Disease](https://mlsanity.com/paper/2603.22225/) (2026)
- [Training-Free Cross-Lingual Dysarthria Severity Assessment via Phonological Subspace Analysis in Self-Supervised Speech Representations](https://mlsanity.com/paper/2604.10123/) (2026)
- [Language Orthogonalization for Zero-Shot Cross-Lingual Audio Deepfake Detection](https://mlsanity.com/paper/2609.16458/) (2026)
- [Phonological Subspace Collapse Is Aetiology-Specific and Cross-Lingually Stable: Evidence from 3,374 Speakers](https://mlsanity.com/paper/2604.21706/) (2026)
- [Phone Segmentation and Recognition through Phonological Activation Mapping](https://mlsanity.com/paper/2607.09020/) (2026)
- [Mitigating Accent-Language Confusion in Self-Supervised Speech Representations for Language Identification](https://mlsanity.com/paper/2610.09486/) (2026)
- [Deep Speech Representations Track Word Timing along a Low-Dimensional Circular Trajectory](https://mlsanity.com/paper/2610.16273/) (2026)

## Abstract (arXiv)

Transformer-based self-supervised speech models (S3Ms) are often described as contextualized, yet what this entails remains unclear. Here, we focus on how a single frame-level S3M representation can encode phones and their surrounding context. Prior work has shown that S3Ms represent phones compositionally; for example, phonological vectors such as voicing, bilabiality, and nasality vectors are superposed in the S3M representation of [m]. We extend this view by proposing that phonological information from a sequence of neighboring phones is also compositionally encoded in a single frame, such that vectors corresponding to previous, current, and next phones are superposed within a single frame-level representation. We show that this structure has several properties, including orthogonality between relative positions, and emergence of implicit phonetic boundaries. Together, our findings advance our understanding of context-dependent S3M representations.