mlsanityData to 7 Oct 2026

Do learned speech symbols follow Zipf's law?

arXiv 2309.09690 ↗First posted 18 Sep 2023cs.CL cs.SD eess.AScited by 3

Not on the map. Why?

Abstract (arXiv)

In this study, we investigate whether speech symbols, learned through deep learning, follow Zipf's law, akin to natural language symbols. Zipf's law is an empirical law that delineates the frequency distribution of words, forming fundamentals for statistical analysis in natural language processing. Natural language symbols, which are invented by humans to symbolize speech content, are recognized to comply with this law. On the other hand, recent breakthroughs in spoken language processing have given rise to the development of learned speech symbols; these are data-driven symbolizations of speech content. Our objective is to ascertain whether these data-driven speech symbols follow Zipf's law, as the same as natural language symbols. Through our investigation, we aim to forge new ways for the statistical analysis of spoken language processing.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 13 of its 29 references that are in the map. First posted on arXiv 18 Sep 2023.

Builds on

13 of 29 references.

References · 13

  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  4. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  5. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 611
  6. 2021Generative Spoken Language Modeling from Raw Audiocited by 336
  7. 2016Multi-Agent Cooperation and the Emergence of (Natural) Languagecited by 181
  8. 2017Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbolscited by 123
  9. 2021BYOL for Audio: Self-Supervised Learning for General-Purpose Audio Representationcited by 106
  10. 2017Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialogcited by 86
  11. 2019Emergent Linguistic Phenomena in Multi-Agent Communication Gamescited by 36
  12. 2022J-MAC: Japanese multi-speaker audiobook corpus for speech synthesiscited by 4
1 more
  1. 2023How Generative Spoken Language Modeling Encodes Noisy Speech: Investigation from Phonetics to Syntacticscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Analysing the Language of Neural Audio Codecscited by 2
  2. 2026Language-Statistical Analysis of Neural Audio Codec Tokens Across Architectures, Corpora, and Noise Conditionscited by 1
  3. 2025A Knowledge-Based Language Model: Deducing Grammatical Knowledge in a Multi-Agent Language Acquisition Simulationcited by 0