# Sparks of Large Audio Models: A Survey and Outlook

arXiv: https://arxiv.org/abs/2308.12792 · HTML: https://mlsanity.com/paper/2308.12792/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Aug 2023.

## Built on by (most cited first, 30 of 41)

- [Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontiers](https://mlsanity.com/paper/2506.23918/) (2025)
- [On The Landscape of Spoken Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2504.08528/) (2025)
- [Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook](https://mlsanity.com/paper/2310.10196/) (2023)
- [A Comprehensive Survey of Hallucination in Large Language, Image, Video and Audio Foundation Models](https://mlsanity.com/paper/2405.09589/) (2024)
- [Recent Advances in Speech Language Models: A Survey](https://mlsanity.com/paper/2410.03751/) (2024)
- [BLSP: Bootstrapping Language-Speech Pre-training via Behavior Alignment of Continuation Writing](https://mlsanity.com/paper/2309.00916/) (2023)
- [Audio-CoT: Exploring Chain-of-Thought Reasoning in Large Audio Language Model](https://mlsanity.com/paper/2501.07246/) (2025)
- [Foundation Models for Music: A Survey](https://mlsanity.com/paper/2408.14340/) (2024)
- [Image Segmentation in Foundation Model Era: A Survey](https://mlsanity.com/paper/2408.12957/) (2024)
- [Next Token Prediction Towards Multimodal Intelligence: A Comprehensive Survey](https://mlsanity.com/paper/2412.18619/) (2024)
- [Speech Translation with Speech Foundation Models and Large Language Models: What is There and What is Missing?](https://mlsanity.com/paper/2402.12025/) (2024)
- [XY-Tokenizer: Mitigating the Semantic-Acoustic Conflict in Low-Bitrate Speech Codecs](https://mlsanity.com/paper/2506.23325/) (2025)
- [Surveying the MLLM Landscape: A Meta-Review of Current Surveys](https://mlsanity.com/paper/2409.18991/) (2024)
- [How "Real" is Your Real-Time Simultaneous Speech-to-Text Translation System?](https://mlsanity.com/paper/2412.18495/) (2024)
- [MOSS-TTS Technical Report](https://mlsanity.com/paper/2603.18090/) (2026)
- [Game-Time: Evaluating Temporal Dynamics in Spoken Language Models](https://mlsanity.com/paper/2509.26388/) (2025)
- [Slamming: Training a Speech Language Model on One GPU in a Day](https://mlsanity.com/paper/2502.15814/) (2025)
- [Learning to Inference Adaptively for Multimodal Large Language Models](https://mlsanity.com/paper/2503.10905/) (2025)
- [When Large Language Models Meet Speech: A Survey on Integration Approaches](https://mlsanity.com/paper/2502.19548/) (2025)
- [Zero-AVSR: Zero-Shot Audio-Visual Speech Recognition with LLMs by Learning Language-Agnostic Speech Representations](https://mlsanity.com/paper/2503.06273/) (2025)
- [Mind the Gap! Static and Interactive Evaluations of Large Audio Models](https://mlsanity.com/paper/2502.15919/) (2025)
- [Hearing to Translate: The Effectiveness of Speech Modality Integration into LLMs](https://mlsanity.com/paper/2512.16378/) (2025)
- [Pheme: Efficient and Conversational Speech Generation](https://mlsanity.com/paper/2401.02839/) (2024)
- [SPES: Spectrogram Perturbation for Explainable Speech-to-Text Generation](https://mlsanity.com/paper/2411.01710/) (2024)
- [Prepending or Cross-Attention for Speech-to-Text? An Empirical Comparison](https://mlsanity.com/paper/2501.02370/) (2025)
- [Negation Blindness in Large Language Models: Unveiling the NO Syndrome in Image Generation](https://mlsanity.com/paper/2409.00105/) (2024)
- [Survey of End-to-End Multi-Speaker Automatic Speech Recognition for Monaural Audio](https://mlsanity.com/paper/2505.10975/) (2025)
- [Breaking Audio Large Language Models by Attacking Only the Encoder: A Universal Targeted Latent-Space Audio Attack](https://mlsanity.com/paper/2512.23881/) (2025)
- [Via Score to Performance: Efficient Human-Controllable Long Song Generation with Bar-Level Symbolic Notation](https://mlsanity.com/paper/2508.01394/) (2025)
- [The Unheard Alternative: Contrastive Explanations for Speech-to-Text Models](https://mlsanity.com/paper/2509.26543/) (2025)

## Abstract (arXiv)

This survey paper provides a comprehensive overview of the recent advancements and challenges in applying large language models to the field of audio signal processing. Audio processing, with its diverse signal representations and a wide range of sources--from human voices to musical instruments and environmental sounds--poses challenges distinct from those found in traditional Natural Language Processing scenarios. Nevertheless, textitLarge Audio Models, epitomized by transformer-based architectures, have shown marked efficacy in this sphere. By leveraging massive amount of data, these models have demonstrated prowess in a variety of audio tasks, spanning from Automatic Speech Recognition and Text-To-Speech to Music Generation, among others. Notably, recently these Foundational Audio Models, like SeamlessM4T, have started showing abilities to act as universal translators, supporting multiple speech tasks for up to 100 languages without any reliance on separate task-specific systems. This paper presents an in-depth analysis of state-of-the-art methodologies regarding textitFoundational Large Audio Models, their performance benchmarks, and their applicability to real-world scenarios. We also highlight current limitations and provide insights into potential future research directions in the realm of textitLarge Audio Models with the intent to spark further discussion, thereby fostering innovation in the next generation of audio-processing systems. Furthermore, to cope with the rapid development in this area, we will consistently update the relevant repository with relevant recent articles and their open-source implementations at https://github.com/EmulationAI/awesome-large-audio-models.