mlsanityData to 7 Oct 2026

Towards audio language modeling -- an overview

arXiv 2402.13236 ↗First posted 20 Feb 2024eess.AS cs.SDcited by 40

Not on the map. Why?

Abstract (arXiv)

Neural audio codecs are initially introduced to compress audio data into compact codes to reduce transmission latency. Researchers recently discovered the potential of codecs as suitable tokenizers for converting continuous audio into discrete codes, which can be employed to develop audio language models (LMs). Numerous high-performance neural audio codecs and codec-based LMs have been developed. The paper aims to provide a thorough and systematic overview of the neural audio codec models and codec-based LMs.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 Feb 2024.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 22; 2026: 8.

40 papers.

Sort:

By area:

  1. 2025On The Landscape of Spoken Language Models: A Comprehensive Surveycited by 116
  2. 2025Audio-CoT: Exploring Chain-of-Thought Reasoning in Large Audio Language Modelcited by 60
  3. 2024Baichuan-Omni Technical Reportcited by 42
  4. 2025Recent Advances in Discrete Speech Tokens: A Reviewcited by 42
  5. 2025MegaTTS 3: Sparse Alignment Enhanced Latent Diffusion Transformer for Zero-Shot Speech Synthesiscited by 39
  6. 2025Towards Holistic Evaluation of Large Audio-Language Models: A Comprehensive Surveycited by 38
  7. 2024Autoregressive Diffusion Transformer for Text-to-Speech Synthesiscited by 34
  8. 2024Foundation Models for Music: A Surveycited by 34
  9. 2024DASB - Discrete Audio and Speech Benchmarkcited by 32
  10. 2025Inference-Time Computations for LLM Reasoning and Planning: A Benchmark and Insightscited by 30
  11. 2025Creativity in LLM-based Multi-Agent Systems: A Surveycited by 23
  12. 2024Preference Alignment Improves Language Model-Based TTScited by 21
28 more
  1. 2025XY-Tokenizer: Mitigating the Semantic-Acoustic Conflict in Low-Bitrate Speech Codecscited by 21
  2. 2024Language in Vivo vs. in Silico: Size Matters but Larger Language Models Still Do Not Comprehend Language on a Par with Humans Due to Impenetrable Semantic Referencecited by 15
  3. 2025Game-Time: Evaluating Temporal Dynamics in Spoken Language Modelscited by 13
  4. 2026MOSS-TTS Technical Reportcited by 10
  5. 2025Live Music Modelscited by 9
  6. 2024DC-Spin: A Speaker-invariant Speech Tokenizer for Spoken Language Modelscited by 8
  7. 2025GenVC: Self-Supervised Zero-Shot Voice Conversioncited by 8
  8. 2025From Alignment to Advancement: Bootstrapping Audio-Language Alignment with Synthetic Datacited by 7
  9. 2026AQUA-Bench: Beyond Finding Answers to Knowing When There Are None in Audio Question Answeringcited by 6
  10. 2025Layer-wise Minimal Pair Probing Reveals Contextual Grammatical-Conceptual Hierarchy in Speech Representationscited by 5
  11. 2025Learning Interpretable Features in Audio Latent Spaces via Sparse Autoencoderscited by 5
  12. 2025Learning Sparsity for Effective and Efficient Music Performance Question Answeringcited by 4
  13. 2026Walking Through Uncertainty: An Empirical Study of Uncertainty Estimation for Audio-Aware Large Language Modelscited by 4
  14. 2024Continuous Speech Tokenizer in Text To Speechcited by 3
  15. 2024A Closer Look at Neural Codec Resynthesis: Bridging the Gap between Codec and Waveform Generationcited by 3
  16. 2025CantoASR: Prosody-Aware ASR-LALM Collaboration for Low-Resource Cantonesecited by 3
  17. 2024VQalAttent: a Transparent Speech Generation Pipeline based on Transformer-learned VQ-VAE Latent Spacecited by 2
  18. 2025Analysing the Language of Neural Audio Codecscited by 2
  19. 2025Towards Efficient Speech-Text Jointly Decoding within One Speech Language Modelcited by 1
  20. 2026Aligning Paralinguistic Understanding and Generation in Speech LLMs via Multi-Task Reinforcement Learningcited by 1
  21. 2026The Sound of Absence: Audio-Language Embedding Models Struggle with Negationcited by 1
  22. 2026Language-Statistical Analysis of Neural Audio Codec Tokens Across Architectures, Corpora, and Noise Conditionscited by 1
  23. 2025Towards Generalized Source Tracing for Codec-Based Deepfake Speechcited by 0
  24. 2025Task-Specific Audio Coding for Machines: Machine-Learned Latent Features Are Codes for That Machinecited by 0
  25. 2025Testing chatbots on the creation of encoders for audio conditioned image generationcited by 0
  26. 2025MI-Fuse: Label Fusion for Unsupervised Domain Adaptation with Closed-Source Large-Audio Language Modelcited by 0
  27. 2026MoVE: Translating Laughter and Tears via Mixture of Vocalization Experts in Speech-to-Speech Translationcited by 0
  28. 2026PolySpeech-100: A Large-Scale Benchmark for Speech Understanding Across 100+ Languages and Dialectscited by 0