mlsanityData to 7 Oct 2026

MERaLiON-AudioLLM: Bridging Audio and Language with Large Language Models

arXiv 2412.09818 ↗First posted 13 Dec 2024cs.CL cs.AIcited by 7

On the map: LLM reasoning → Audio-language model reasoning

Abstract (arXiv)

We introduce MERaLiON-AudioLLM (Multimodal Empathetic Reasoning and Learning in One Network), the first speech-text model tailored for Singapore's multilingual and multicultural landscape. Developed under the National Large Language Models Funding Initiative, Singapore, MERaLiON-AudioLLM integrates advanced speech and text processing to address the diverse linguistic nuances of local accents and dialects, enhancing accessibility and usability in complex, multilingual environments. Our results demonstrate improvements in both speech recognition and task-specific understanding, positioning MERaLiON-AudioLLM as a pioneering solution for region specific AI applications. We envision this release to set a precedent for future models designed to address localised linguistic and cultural contexts in a global framework.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning. Builds on 47 of its 58 references that are in the map. First posted on arXiv 13 Dec 2024.

Builds on

47 of 58 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Mini-Omni: Language Models Can Hear, Talk While Thinking in Streaming29 Aug
  2. 2024GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilities17 Jun
  3. 2023AudioChatLlama: Towards General-Purpose Speech Abilities for LLMs12 Nov

Other references · 44

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  2. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  4. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  5. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  6. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  7. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 935
  8. 2017VoxCeleb: a large-scale speaker identification datasetcited by 714
  9. 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 636
  10. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 591
  11. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 536
  12. 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 518
32 more
  1. 2024Large Language Models: A Surveycited by 506
  2. 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 492
  3. 2022BEATs: Audio Pre-Training with Acoustic Tokenizerscited by 378
  4. 2023AudioPaLM: A Large Language Model That Can Speak and Listencited by 333
  5. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 288
  6. 2023Pengi: An Audio Language Model for Audio Taskscited by 217
  7. 2023Listen, Think, and Understandcited by 206
  8. 2024Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilitiescited by 183
  9. 2024LLaMA-Omni: Seamless Speech Interaction with Large Language Modelscited by 169
  10. 2023On decoder-only architecture for speech-to-text and large language model integrationcited by 144
  11. 2024AudioBench: A Universal Benchmark for Audio Large Language Modelscited by 124
  12. 2024Spirit LM: Interleaved Spoken and Written Language Modelcited by 123
  13. 2024FunAudioLLM: Voice Understanding and Generation Foundation Models for Natural Interaction Between Humans and LLMscited by 118
  14. 2024WavLLM: Towards Robust and Adaptive Speech Large Language Modelcited by 107
  15. 2023Joint Audio and Speech Understandingcited by 99
  16. 2024WavChat: A Survey of Spoken Dialogue Modelscited by 88
  17. 2023VioLA: Unified Codec Language Models for Speech Recognition, Synthesis, and Translationcited by 87
  18. 2024An Embarrassingly Simple Approach for LLM with Strong ASR Capacitycited by 86
  19. 2023Connecting Speech Encoder and Large Language Model for ASRcited by 79
  20. 2018Spoken SQuAD: A Study of Mitigating the Impact of Speech Recognition Errors on Listening Comprehensioncited by 74
  21. 2021Earnings-21: A Practical Benchmark for ASR in the Wildcited by 60
  22. 2022Earnings-22: A Practical Benchmark for Accents in the Wildcited by 46
  23. 2024EMOVA: Empowering Language Models to See, Hear and Speak with Vivid Emotionscited by 44
  24. 2021ConvMLP: Hierarchical Convolutional MLPs for Visioncited by 37
  25. 2024Distilling an End-to-End Voice Assistant Without Instruction Training Datacited by 35
  26. 2022SLUE Phase-2: A Benchmark Suite of Diverse Spoken Language Understanding Taskscited by 34
  27. 2024DeSTA: Enhancing Speech Language Models through Descriptive Speech-Text Alignmentcited by 32
  28. 2024BESTOW: Efficient and Streamable Speech Language Model with the Best of Two Worlds in GPT and T5cited by 18
  29. 2024Frozen Large Language Models Can Perceive Paralinguistic Aspects of Speechcited by 18
  30. 2024Comparing Discrete and Continuous Space LLMs for Speech Recognitioncited by 15
  31. 2024LLaST: Improved End-to-end Speech Translation System Leveraged by Large Language Modelscited by 12
  32. 2024MoWE-Audio: Multitask AudioLLMs with Mixture of Weak Encoderscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 3.

7 papers.

  1. 2025Benchmarking Contextual and Paralinguistic Reasoning in Speech-LLMs: A Case Study with In-the-Wild Datacited by 10
  2. 2025Summarizing Speech: A Comprehensive Surveycited by 5
  3. 2025AsyncSwitch: Asynchronous Text-Speech Adaptation for Code-Switched ASRcited by 1
  4. 2026Greater accessibility can amplify discrimination in generative AIcited by 1
  5. 2026I'll Keep an Ear Out: Teaching AudioLLMs Proactive Audio Assistancecited by 1
  6. 2025Bi-directional Context-Enhanced Speech Large Language Models for Multilingual Conversational ASRcited by 0
  7. 2026Emotion Across Speech and Faces: Shared Affective Mechanisms in Multimodal Foundation Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024MERaLiON-SpeechEncoder: Towards a Speech Foundation Model for Singapore and Beyond
  2. 2026Efficiently Adapting Spoken Language Models for the Singaporean Context
  3. 2025MERaLiON-SER: Robust Speech Emotion Recognition Model for English and SEA Languages
  4. 2025Mellow: a small audio language model for reasoning
  5. 2026Multilingual and Multimodal LLMs in the Wild: Building for Low-Resource Languages
  6. 2024MERaLiON-TextLLM: Cross-Lingual Understanding of Large Language Models in Chinese, Indonesian, Malay, and Singlish
  7. 2025Multimodal Large Language Models Meet Multimodal Emotion Recognition and Reasoning: A Survey
  8. 2026SEA-SpeechBench: A Large-Scale Multitask Benchmark for Speech Understanding Across Southeast Asia