mlsanityData to 7 Oct 2026

The Zero Resource Speech Challenge 2021: Spoken language modelling

arXiv 2104.14700 ↗First posted 29 Apr 2021cs.CL cs.AIcited by 47

Not on the map. Why?

Abstract (arXiv)

We present the Zero Resource Speech Challenge 2021, which asks participants to learn a language model directly from audio, without any text or labels. The challenge is based on the Libri-light dataset, which provides up to 60k hours of audio from English audio books without any associated text. We provide a pipeline baseline system consisting on an encoder based on contrastive predictive coding (CPC), a quantizer (k-means) and a standard language model (BERT or LSTM). The metrics evaluate the learned representations at the acoustic (ABX discrimination), lexical (spot-the-word), syntactic (acceptability judgment) and semantic levels (similarity judgment). We present an overview of the eight submitted systems from four groups and discuss the main results.

Key facts. Cited by 47 papers in the map (data through 7 Oct 2026). Builds on 18 of its 33 references that are in the map. First posted on arXiv 29 Apr 2021.

Builds on

18 of 33 references.

References · 18

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  5. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  6. 2019BLiMP: The Benchmark of Linguistic Minimal Pairs for Englishcited by 553
  7. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 462
  8. 2021Generative Spoken Language Modeling from Raw Audiocited by 336
  9. 2019vq-wav2vec: Self-Supervised Learning of Discrete Speech Representationscited by 335
  10. 2019Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoderscited by 196
  11. 2014SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimationcited by 170
  12. 2020Unsupervised pretraining transfers well across languagescited by 100
6 more
  1. 2020The Zero Resource Speech Benchmark 2021: Metrics and baselines for unsupervised spoken language modelingcited by 100
  2. 2018Speech2Vec: A Sequence-to-Sequence Framework for Learning Word Embeddings from Speechcited by 52
  3. 2016SimVerb-3500: A Large-Scale Evaluation Set of Verb Similaritycited by 44
  4. 2020The Zero Resource Speech Challenge 2020: Discovering discrete subword and word unitscited by 39
  5. 2019The Zero Resource Speech Challenge 2019: TTS without Tcited by 38
  6. 2017Replicability Analysis for Natural Language Processing: Testing Significance with Multiple Datasetscited by 32

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 8; 2023: 12; 2024: 7; 2025: 10; 2026: 10.

47 papers.

Sort:

  1. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 611
  2. 2025On The Landscape of Spoken Language Models: A Comprehensive Surveycited by 116
  3. 2022ContentVec: An Improved Self-Supervised Speech Representation by Disentangling Speakerscited by 73
  4. 2024Self-Supervised Speech Representations are More Phonetic than Semanticcited by 70
  5. 2022Self-supervised language learning from raw audio: Lessons from the Zero Resource Speech Challengecited by 34
  6. 2023BabySLM: language-acquisition-friendly benchmark of self-supervised spoken language modelscited by 29
  7. 2022Do self-supervised speech models develop human-like perception biases?cited by 28
  8. 2024Speech-Copilot: Leveraging Large Language Models for Speech Processing via Task Decomposition, Modularization, and Program Generationcited by 27
  9. 2022Are discrete units necessary for Spoken Language Modeling?cited by 24
  10. 2023ProsAudit, a prosodic benchmark for self-supervised speech modelscited by 24
  11. 2024Salmon: A Suite for Acoustic Language Model Evaluationcited by 24
  12. 2023Putting Natural in Natural Language Processingcited by 17
35 more
  1. 2023Zero Resource Code-switched Speech Benchmark Using Speech Utterance Pairs For Multiple Spoken Languagescited by 12
  2. 2025Slamming: Training a Speech Language Model on One GPU in a Daycited by 11
  3. 2022A Brief Overview of Unsupervised Neural Speech Representation Learningcited by 9
  4. 2023SpeechGLUE: How Well Can Self-Supervised Speech Models Capture Linguistic Knowledge?cited by 9
  5. 2022MTI-Net: A Multi-Target Speech Intelligibility Prediction Modelcited by 6
  6. 2025IPA-CHILDES & G2P+: Feature-Rich Resources for Cross-Lingual Phonology and Phonemic Language Modelingcited by 6
  7. 2025SpidR: Learning Fast and Stable Linguistic Units for Spoken Language Models Without Supervisioncited by 6
  8. 2023Zero-Shot Automatic Pronunciation Assessmentcited by 5
  9. 2025Spoken Language Modeling with Duration-Penalized Self-Supervised Unitscited by 5
  10. 2026ZeroSyl: Simple Zero-Resource Syllable Tokenization for Spoken Language Modelingcited by 5
  11. 2026An Empirical Recipe for Universal Phone Recognitioncited by 5
  12. 2023Exploring How Generative Adversarial Networks Learn Phonological Representationscited by 4
  13. 2022Bootstrapping meaning through listening: Unsupervised learning of spoken sentence embeddingscited by 2
  14. 2023Cascading and Direct Approaches to Unsupervised Constituency Parsing on Spoken Sentencescited by 2
  15. 2023ZRIGF: An Innovative Multimodal Framework for Zero-Resource Image-Grounded Dialogue Generationcited by 2
  16. 2023Enhance audio generation controllability through representation similarity regularizationcited by 2
  17. 2025BabyLM's First Words: Word Segmentation as a Phonological Probing Taskcited by 2
  18. 2025Scaling Spoken Language Models with Syllabic Speech Tokenizationcited by 2
  19. 2023Representation Learning With Hidden Unit Clustering For Low Resource Speech Applicationscited by 1
  20. 2023Efficiency-oriented approaches for self-supervised speech representation learningcited by 1
  21. 2024Removing Speaker Information from Speech Representation using Variable-Length Soft Poolingcited by 1
  22. 2024Towards the Next Frontier in Speech Representation Learning Using Disentanglementcited by 1
  23. 2024Analytic Study of Text-Free Speech Synthesis for Raw Audio using a Self-Supervised Learning Modelcited by 1
  24. 2025Factorized RVQ-GAN For Disentangled Speech Tokenizationcited by 1
  25. 2025SpidR-Adapt: A Universal Speech Representation Model for Few-Shot Adaptationcited by 1
  26. 2026WavSLM: Single-Stream Speech Language Modeling via WavLM Distillationcited by 1
  27. 2026Scaling Properties of Continuous Diffusion Spoken Language Modelscited by 1
  28. 2026PashtoTTS-Bench: automated screening for low-resource non-Latin-script text-to-speechcited by 1
  29. 2026Connecting Speech to Words through Imagescited by 1
  30. 2024Learning Semantic Information from Raw Audio Signal Using Both Contextual and Phonetic Representationscited by 0
  31. 2025Cross-Lingual Interleaving for Speech Language Modelscited by 0
  32. 2026On the Fallacy of Global Token Perplexity in Spoken Language Model Evaluationcited by 0
  33. 2026DiscoPhon: Benchmarking the Unsupervised Discovery of Phoneme Inventories With Discrete Speech Unitscited by 0
  34. 2026Mapping Written Words to Spoken Words in a Different Language Using Only Visual Groundingcited by 0
  35. 2026Benchmarking_Fast_Domain_Adaptation_for_Unsupervised_Speech_Unitscited by 0