mlsanityData to 14 Oct 2026

Wave to Syntax: Probing spoken language models for syntax

arXiv 2305.18957 ↗First posted 30 May 2023cs.CLcited by 18

Not on the map. Why?

Abstract (arXiv)

Understanding which information is encoded in deep models of spoken and written language has been the focus of much research in recent years, as it is crucial for debugging and improving these architectures. Most previous work has focused on probing for speaker characteristics, acoustic and phonological information in models of spoken language, and for syntactic information in models of written language. Here we focus on the encoding of syntax in several self-supervised and visually grounded models of spoken language. We employ two complementary probing methods, combined with baselines and reference representations to quantify the degree to which syntactic structure is encoded in the activations of the target models. We show that syntax is captured most prominently in the middle layers of the networks, and more explicitly within models with more parameters.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 21 of its 32 references that are in the map. First posted on arXiv 30 May 2023.

Builds on

21 of 32 references.

References · 21

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  4. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  5. 2020A Primer in BERTology: What we know about how BERT workscited by 815
  6. 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 723
  7. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 457
  8. 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 298
  9. 2018Analysis Methods in Neural Language Processing: A Surveycited by 277
  10. 2019Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoderscited by 198
  11. 2021Neuron-level Interpretation of Deep NLP Models: A Surveycited by 81
  12. 2020Text-Free Image-to-Speech Synthesis Using Learned Segmental Unitscited by 51
9 more
  1. 2019Correlating neural and symbolic representations of languagecited by 33
  2. 2022Probing phoneme, language and speaker information in unsupervised speech representationscited by 33
  3. 2017Analyzing Hidden Representations in End-to-End Automatic Speech Recognition Systemscited by 32
  4. 2021Visually grounded models of spoken language: A survey of datasets, architectures and evaluation techniquescited by 32
  5. 2017Encoding of phonology in a recurrent neural model of grounded speechcited by 27
  6. 2022Self-Supervised Representation Learning for Speech Using Visual Grounding and Masked Language Modelingcited by 25
  7. 2019Analyzing Phonetic and Graphemic Representations in End-to-End Automatic Speech Recognitioncited by 16
  8. 2020Analyzing analytical methods: The case of phonology in neural models of spoken languagecited by 16
  9. 2021Can phones, syllables, and words emerge as side-products of cross-situational audiovisual learning? -- A computational investigationcited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 6; 2025: 4; 2026: 5.

18 papers.

Sort:

  1. 2024Self-Supervised Speech Representations are More Phonetic than Semanticcited by 71
  2. 2023What Do Self-Supervised Speech Models Know About Words?cited by 64
  3. 2024Encoding of lexical tone in self-supervised models of spoken languagecited by 23
  4. 2024Human-like Linguistic Biases in Neural Speech Models: Phonetic Categorization and Phonotactic Constraints in Wav2Vec2.0cited by 19
  5. 2023Homophone Disambiguation Reveals Patterns of Context Mixing in Speech Transformerscited by 13
  6. 2026[b] = [d] - [t] + [p]: Self-supervised Speech Models Discover Phonological Vector Arithmeticcited by 12
  7. 2026Self-Supervised Speech Models Encode Phonetic Context via Position-dependent Orthogonal Subspacescited by 8
  8. 2025Layer-wise Minimal Pair Probing Reveals Contextual Grammatical-Conceptual Hierarchy in Speech Representationscited by 5
  9. 2024Perception of Phonological Assimilation by Neural Speech Recognition Modelscited by 3
  10. 2025Analyzing the relationships between pretraining language, phonetic, tonal, and speaker information in self-supervised speech modelscited by 3
  11. 2025Word stress in self-supervised speech models: A cross-linguistic comparisoncited by 3
  12. 2024What has LeBenchmark Learnt about French Syntax?cited by 2
6 more
  1. 2023Few-Shot Spoken Language Understanding via Joint Speech-Text Modelscited by 1
  2. 2025A Linguistically Motivated Analysis of Intonational Phrasing in Text-to-Speech Systems: Revealing Gaps in Syntactic Sensitivitycited by 1
  3. 2026Tracking the emergence of linguistic structure in self-supervised models learning from speechcited by 1
  4. 2024Investigating large language models for their competence in extracting grammatically sound sentences from transcribed noisy utterancescited by 0
  5. 2026Beyond Decodability: Reconstructing Language Model Representations with an Encoding Probecited by 0
  6. 2026Do speech foundation models really learn words?cited by 0