mlsanityData to 14 Oct 2026

Analysing the Language of Neural Audio Codecs

arXiv 2509.01390 ↗First posted 1 Sep 2025cs.CL eess.AScited by 2

Not on the map. Why?

Abstract (arXiv)

This study presents a comparative analysis of the statistical and linguistic properties of neural audio codecs (NACs). We investigate discrete speech tokens produced by various NAC models, examining their adherence to linguistic statistical laws such as Zipf's law and Heaps' law, as well as their entropy and redundancy. To assess how these token-level properties relate to semantic and acoustic preservation in synthesized speech, we evaluate intelligibility using error rates of automatic speech recognition, and quality using the UTMOS score. Our results reveal that NAC tokens, particularly 3-grams, exhibit language-like statistical patterns. Moreover, these properties, together with measures of information content, are found to correlate with improved performances in speech recognition and resynthesis tasks. These findings offer insights into the structure of NAC token sequences and inform the design of more effective generative speech models.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 28 of its 41 references that are in the map. First posted on arXiv 1 Sep 2025.

Builds on

28 of 41 references.

References · 28

  1. 2023Visual Instruction Tuningcited by 7,869
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  4. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  5. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  6. 2021SoundStream: An End-to-End Neural Audio Codeccited by 771
  7. 2022High Fidelity Neural Audio Compressioncited by 687
  8. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
  9. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 616
  10. 2023High-Fidelity Audio Compression with Improved RVQGANcited by 431
  11. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 390
  12. 2017AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baselinecited by 373
16 more
  1. 2020FSD50K: An Open Dataset of Human-Labeled Sound Eventscited by 342
  2. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 260
  3. 2023HiFi-Codec: Group-residual Vector quantization for High Fidelity Audio Codeccited by 97
  4. 2023SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Modelscited by 87
  5. 2022ICASSP 2022 Deep Noise Suppression Challengecited by 60
  6. 2019CSS10: A Collection of Single Speaker Speech Datasets for 10 Languagescited by 54
  7. 2023Analysing Discrete Self Supervised Speech Representation for Spoken Language Modelingcited by 47
  8. 2023AudioDec: An Open-source Streaming High-fidelity Neural Audio Codeccited by 47
  9. 2023FunCodec: A Fundamental, Reproducible and Integrable Open-source Toolkit for Neural Speech Codeccited by 44
  10. 2025Recent Advances in Discrete Speech Tokens: A Reviewcited by 43
  11. 2024Towards audio language modeling -- an overviewcited by 40
  12. 2024Codec-SUPERB: An In-Depth Analysis of Sound Codec Modelscited by 26
  13. 2024Analyzing The Language of Visual Tokenscited by 12
  14. 2024Codec-ASR: Training Performant Automatic Speech Recognition Systems with Discrete Speech Representationscited by 8
  15. 2024Towards Audio Codec-based Speech Separationcited by 4
  16. 2023Do learned speech symbols follow Zipf's law?cited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Do Neural Codecs Generalize? A Controlled Study Across Unseen Languages and Non-Speech Taskscited by 1
  2. 2026Language-Statistical Analysis of Neural Audio Codec Tokens Across Architectures, Corpora, and Noise Conditionscited by 1