Do Audio Language Models Hear and Read Distinctive Features Alike?
Not on the map. Why?
Abstract (arXiv)
Audio language models pass speech and text through a single decoder. We ask whether that decoder represents a distinctive feature in the same direction when a phoneme is heard and when it is read. For minimal pairs of phonemes differing in one feature, we take the offset between the two members' mean representations. Averaging those offsets gives a direction for each stream, and we measure the cosine between the two. Because the two streams already agree about arbitrary phoneme pairs, we compare every measure against a reference built from random pairings rather than against zero. We apply this to 6 models, 7 features and 15 languages from 11 families. Only voicing in the two Qwen2.5-Omni models exceeds that reference after correction for multiple testing, and the reference varies by a factor of seven between models. In three of the six models, voicing has one direction in audio across the 14 languages with enough minimal pairs to measure it, and every language pair agrees in two of them. The model family, not the model size, predicts which stream represents a feature.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 12 of its 17 references that are in the map. First posted on arXiv 24 Sep 2026.
Builds on
12 of 17 references.
References · 12
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
- 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 784
- 2025Qwen2.5-Omni Technical Reportcited by 736
- 2019Designing and Interpreting Probes with Control Taskscited by 640
- 2024Qwen2-Audio Technical Reportcited by 560
- 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 501
- 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 298
- 2026Gemma 4 Technical Reportcited by 259
- 2025Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Modelscited by 219
- 2021All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Qualitycited by 156
- 2026[b] = [d] - [t] + [p]: Self-supervised Speech Models Discover Phonological Vector Arithmeticcited by 12
- 2025ALAS: An Automatic Latent Alignment Score for Audio Language Modelscited by 3
Built on by
Not cited yet.