mlsanityData to 14 Oct 2026

What do self-supervised speech models know about Dutch? Analyzing advantages of language-specific pre-training

arXiv 2506.00981 ↗First posted 1 Jun 2025cs.CL cs.AI cs.SDcited by 4

Not on the map. Why?

Abstract (arXiv)

How language-specific are speech representations learned by self-supervised models? Existing work has shown that a range of linguistic features can be successfully decoded from end-to-end models trained only on speech recordings. However, it's less clear to what extent pre-training on specific languages improves language-specific linguistic information. Here we test the encoding of Dutch phonetic and lexical information in internal representations of self-supervised Wav2Vec2 models. Pre-training exclusively on Dutch improves the representation of Dutch linguistic features as compared to pre-training on similar amounts of English or larger amounts of multilingual data. This language-specific advantage is well-detected by trained clustering or classification probes, and partially observable using zero-shot metrics. Furthermore, the language-specific benefit on linguistic feature encoding aligns with downstream performance on Automatic Speech Recognition.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 20 of its 36 references that are in the map. First posted on arXiv 1 Jun 2025.

Builds on

20 of 36 references.

References · 20

  1. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  2. 2016Enriching Word Vectors with Subword Informationcited by 1,583
  3. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,488
  4. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  5. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 601
  6. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 476
  7. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 463
  8. 2020Unsupervised pretraining transfers well across languagescited by 101
  9. 2016Cognitive Science in the era of Artificial Intelligence: A roadmap for reverse-engineering the infant language-learnercited by 68
  10. 2023What Do Self-Supervised Speech Models Know About Words?cited by 64
  11. 2022Toward a realistic model of speech processing in the brain with self-supervised learningcited by 63
  12. 2022Do self-supervised speech models develop human-like perception biases?cited by 28
8 more
  1. 2023Speech Self-Supervised Representation Benchmarking: Are We Doing it Right?cited by 28
  2. 2024Encoding of lexical tone in self-supervised models of spoken languagecited by 23
  3. 2024Human-like Linguistic Biases in Neural Speech Models: Phonetic Categorization and Phonotactic Constraints in Wav2Vec2.0cited by 19
  4. 2023LeBenchmark 2.0: a Standardized, Replicable and Enhanced Framework for Self-supervised Representations of French Speechcited by 18
  5. 2024A layer-wise analysis of Mandarin and English suprasegmentals in SSL speech modelscited by 18
  6. 2023An Information-Theoretic Analysis of Self-supervised Discrete Representations of Speechcited by 15
  7. 2024On the Encoding of Gender in Transformer-based ASR Representationscited by 4
  8. 2024What has LeBenchmark Learnt about French Syntax?cited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025The Curious Case of Visual Grounding: Different Effects for Speech- and Text-based Language Encoderscited by 1
  2. 2026Tracking the emergence of linguistic structure in self-supervised models learning from speechcited by 1
  3. 2026Beyond Decodability: Reconstructing Language Model Representations with an Encoding Probecited by 0
  4. 2026Perceptual compensation for tonal context in self-supervised speech modelscited by 0