# Evidence of Vocal Tract Articulation in Self-Supervised Learning of Speech

arXiv: https://arxiv.org/abs/2210.11723 · HTML: https://mlsanity.com/paper/2210.11723/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 22 references that are in the map.
- First posted on arXiv 21 Oct 2022.

## Builds on (most cited first, 17 of 17)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language](https://mlsanity.com/paper/2202.03555/) (2022)
- [SUPERB: Speech processing Universal PERformance Benchmark](https://mlsanity.com/paper/2105.01051/) (2021)
- [Libri-Light: A Benchmark for ASR with Limited or No Supervision](https://mlsanity.com/paper/1912.07875/) (2019)
- [vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations](https://mlsanity.com/paper/1910.05453/) (2019)
- [Layer-wise Analysis of a Self-supervised Speech Representation Model](https://mlsanity.com/paper/2107.04734/) (2021)
- [Self-Supervised Speech Representation Learning: A Review](https://mlsanity.com/paper/2205.10643/) (2022)
- [TERA: Self-Supervised Learning of Transformer Encoder Representation for Speech](https://mlsanity.com/paper/2007.06028/) (2020)
- [fairseq S2T: Fast Speech-to-Text Modeling with fairseq](https://mlsanity.com/paper/2010.05171/) (2020)
- [Unsupervised Speech Recognition](https://mlsanity.com/paper/2105.11084/) (2021)
- [DeCoAR 2.0: Deep Contextualized Acoustic Representations with Vector Quantization](https://mlsanity.com/paper/2012.06659/) (2020)
- [What all do audio transformer models hear? Probing Acoustic Representations for Language Delivery and its Structure](https://mlsanity.com/paper/2101.00387/) (2021)
- [Why does Self-Supervised Learning for Speech Recognition Benefit Speaker Recognition?](https://mlsanity.com/paper/2204.12765/) (2022)
- [Probing Acoustic Representations for Phonetic Properties](https://mlsanity.com/paper/2010.13007/) (2020)
- [Deep Speech Synthesis from Articulatory Representations](https://mlsanity.com/paper/2209.06337/) (2022)

## Built on by (most cited first, 30 of 31)

- [Sylber: Syllabic Embedding Representation of Speech from Raw Audio](https://mlsanity.com/paper/2410.07168/) (2024)
- [Coding Speech through Vocal Tract Kinematics](https://mlsanity.com/paper/2406.12998/) (2024)
- [Self-Supervised Models of Speech Infer Universal Articulatory Kinematics](https://mlsanity.com/paper/2310.10788/) (2023)
- [SD-HuBERT: Sentence-Level Self-Distillation Induces Syllabic Organization in HuBERT](https://mlsanity.com/paper/2310.10803/) (2023)
- [[b] = [d] - [t] + [p]: Self-supervised Speech Models Discover Phonological Vector Arithmetic](https://mlsanity.com/paper/2602.18899/) (2026)
- [SLIM: Style-Linguistics Mismatch Model for Generalized Audio Deepfake Detection](https://mlsanity.com/paper/2407.18517/) (2024)
- [Self-Supervised Speech Models Encode Phonetic Context via Position-dependent Orthogonal Subspaces](https://mlsanity.com/paper/2603.12642/) (2026)
- [Improving Speech Inversion Through Self-Supervised Embeddings and Enhanced Tract Variables](https://mlsanity.com/paper/2309.09220/) (2023)
- [Learn from Real: Reality Defender's Submission to ASVspoof5 Challenge](https://mlsanity.com/paper/2410.07379/) (2024)
- [Neural Latent Aligner: Cross-trial Alignment for Learning Representations of Complex, Naturalistic Neural Data](https://mlsanity.com/paper/2308.06443/) (2023)
- [Exploring Pathological Speech Quality Assessment with ASR-Powered Wav2Vec2 in Data-Scarce Context](https://mlsanity.com/paper/2403.20184/) (2024)
- [RT-VC: Real-Time Zero-Shot Voice Conversion with Speech Articulatory Coding](https://mlsanity.com/paper/2506.10289/) (2025)
- [Multimodal Segmentation for Vocal Tract Modeling](https://mlsanity.com/paper/2406.15754/) (2024)
- [Fast, High-Quality and Parameter-Efficient Articulatory Synthesis using Differentiable DSP](https://mlsanity.com/paper/2409.02451/) (2024)
- [Towards disentangling the contributions of articulation and acoustics in multimodal phoneme recognition](https://mlsanity.com/paper/2505.24059/) (2025)
- [emg2speech: Synthesizing speech from electromyography using self-supervised speech models](https://mlsanity.com/paper/2510.23969/) (2025)
- [WavRx: a Disease-Agnostic, Generalizable, and Privacy-Preserving Speech Health Diagnostic Model](https://mlsanity.com/paper/2406.18731/) (2024)
- [Entropy-based Coarse and Compressed Semantic Speech Representation Learning](https://mlsanity.com/paper/2509.00503/) (2025)
- [Sylber 2.0: A Universal Syllable Embedding](https://mlsanity.com/paper/2601.22306/) (2026)
- [Training-Free Cross-Lingual Dysarthria Severity Assessment via Phonological Subspace Analysis in Self-Supervised Speech Representations](https://mlsanity.com/paper/2604.10123/) (2026)
- [Training Articulatory Inversion Models for Interspeaker Consistency](https://mlsanity.com/paper/2505.20529/) (2025)
- [ARTI-6: Towards Six-dimensional Articulatory Speech Encoding](https://mlsanity.com/paper/2509.21447/) (2025)
- [Comparison of sEMG Encoding Accuracy Across Speech Modes Using Articulatory and Phoneme Features](https://mlsanity.com/paper/2604.18920/) (2026)
- [Simulating Articulatory Trajectories with Phonological Feature Interpolation](https://mlsanity.com/paper/2408.04363/) (2024)
- [Enhancing and Exploring Mild Cognitive Impairment Detection with W2V-BERT-2.0](https://mlsanity.com/paper/2501.16201/) (2025)
- [FROST-EMA: Finnish and Russian Oral Speech Dataset of Electromagnetic Articulography Measurements with L1, L2 and Imitated L2 Accents](https://mlsanity.com/paper/2506.08981/) (2025)
- [Speech Emotion Recognition with Phonation Excitation Information and Articulatory Kinematics](https://mlsanity.com/paper/2511.07955/) (2025)
- [Phonological Subspace Collapse Is Aetiology-Specific and Cross-Lingually Stable: Evidence from 3,374 Speakers](https://mlsanity.com/paper/2604.21706/) (2026)
- [What Do Neural Networks Learn for TDOA Estimation? A Cross-Architecture Probing Study](https://mlsanity.com/paper/2606.22020/) (2026)
- [A survey of AI-generated voices and their detection](https://mlsanity.com/paper/2608.15411/) (2026)

## Abstract (arXiv)

Recent self-supervised learning (SSL) models have proven to learn rich representations of speech, which can readily be utilized by diverse downstream tasks. To understand such utilities, various analyses have been done for speech SSL models to reveal which and how information is encoded in the learned representations. Although the scope of previous analyses is extensive in acoustic, phonetic, and semantic perspectives, the physical grounding by speech production has not yet received full attention. To bridge this gap, we conduct a comprehensive analysis to link speech representations to articulatory trajectories measured by electromagnetic articulography (EMA). Our analysis is based on a linear probing approach where we measure articulatory score as an average correlation of linear mapping to EMA. We analyze a set of SSL models selected from the leaderboard of the SUPERB benchmark and perform further layer-wise analyses on two most successful models, Wav2Vec 2.0 and HuBERT. Surprisingly, representations from the recent speech SSL models are highly correlated with EMA traces (best: r = 0.81), and only 5 minutes are sufficient to train a linear model with high performance (r = 0.77). Our findings suggest that SSL models learn to align closely with continuous articulations, and provide a novel insight into speech SSL.