mlsanityData to 14 Oct 2026

Towards Reliable Large Audio Language Model

arXiv 2505.19294 ↗First posted 25 May 2025cs.SD cs.CL cs.HCcited by 3

Not on the map. Why?

Abstract (arXiv)

Recent advancements in large audio language models (LALMs) have demonstrated impressive results and promising prospects in universal understanding and reasoning across speech, music, and general sound. However, these models still lack the ability to recognize their knowledge boundaries and refuse to answer questions they don't know proactively. While there have been successful attempts to enhance the reliability of LLMs, reliable LALMs remain largely unexplored. In this paper, we systematically investigate various approaches towards reliable LALMs, including training-free methods such as multi-modal chain-of-thought (MCoT), and training-based methods such as supervised fine-tuning (SFT). Besides, we identify the limitations of previous evaluation metrics and propose a new metric, the Reliability Gain Index (RGI), to assess the effectiveness of different reliable methods. Our findings suggest that both training-free and training-based methods enhance the reliability of LALMs to different extents. Moreover, we find that awareness of reliability is a "meta ability", which can be transferred across different audio modalities, although significant structural and content differences exist among sound, music, and speech.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 32 of its 54 references that are in the map. First posted on arXiv 25 May 2025.

Builds on

32 of 54 references.

References · 32

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  3. 2023Qwen Technical Reportcited by 3,344
  4. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  5. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
  6. 2022BEATs: Audio Pre-Training with Acoustic Tokenizerscited by 388
  7. 2024MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmarkcited by 288
  8. 2024AIR-Bench: Benchmarking Large Audio-Language Models via Generative Comprehensioncited by 220
  9. 2024Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilitiescited by 186
  10. 2024GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilitiescited by 156
  11. 2023On decoder-only architecture for speech-to-text and large language model integrationcited by 145
  12. 2024Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuningcited by 121
20 more
  1. 2024OmniBench: Towards The Future of Universal Omni-Language Modelscited by 102
  2. 2024Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words?cited by 87
  3. 2023Connecting Speech Encoder and Large Language Model for ASRcited by 80
  4. 2023BLSP: Bootstrapping Language-Speech Pre-training via Behavior Alignment of Continuation Writingcited by 79
  5. 2023LauraGPT: Listen, Attend, Understand, and Regenerate Audio with GPTcited by 78
  6. 2023Music Understanding LLaMA: Advancing Text-to-Music Generation with Question Answering and Captioningcited by 77
  7. 2024ChatMusician: Understanding and Generating Music Intrinsically with LLMcited by 68
  8. 2024Rejection Improves Reliability: Training LLMs to Refuse Unknown Questions Using RL from Knowledge Feedbackcited by 66
  9. 2024Language Model Can Listen While Speakingcited by 63
  10. 2023Prompting Large Language Models for Zero-Shot Domain Adaptation in Speech Recognitioncited by 48
  11. 2023MusiLingo: Bridging Music and Text with Pre-trained Language Models for Music Captioning and Query Responsecited by 45
  12. 2024DeSTA2: Developing Instruction-Following Speech Language Model Without Speech Instruction-Tuning Datacited by 45
  13. 2024MuPT: A Generative Symbolic Music Pretrained Transformercited by 33
  14. 2024DeSTA: Enhancing Speech Language Models through Descriptive Speech-Text Alignmentcited by 33
  15. 2023Paralinguistics-Enhanced Large Language Modeling of Spoken Dialoguecited by 25
  16. 2024Frozen Large Language Models Can Perceive Paralinguistic Aspects of Speechcited by 18
  17. 2024Large Language Model Can Transcribe Speech in Multi-Talker Scenarios with Versatile Instructionscited by 17
  18. 2024Unveiling the Potential of LLM-Based ASR on Chinese Open-Source Datasetscited by 13
  19. 2024DRCap: Decoding CLAP Latents with Retrieval-Augmented Generation for Zero-shot Audio Captioningcited by 11
  20. 2024State-Space Large Audio Language Modelscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Thinking with Sound: Audio Chain-of-Thought Enables Multimodal Reasoning in Large Audio-Language Modelscited by 9
  2. 2026Noise-Aware In-Context Learning for Hallucination Mitigation in ALLMscited by 0
  3. 2026EchoDistill: Robust Large Audio Language Models via Noisy-to-Clean Self-Distillationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Benchmarking and Confidence Evaluation of LALMs For Temporal Reasoning
  2. 2026MUGEN: Evaluating and Improving Multi-audio Understanding of Large Audio-Language Models
  3. 2026Continuous Audio Thinking for Large Audio Language Models
  4. 2024Can Large Audio-Language Models Truly Hear? Tackling Hallucinations with Multi-Task Assessment and Stepwise Audio Reasoning
  5. 2026A Closer Look at Failure Modes in Temporal Understanding of Large Audio-Language Models
  6. 2025DeSTA2.5-Audio: Toward General-Purpose Large Audio Language Model with Self-Generated Cross-Modal Alignment
  7. 2025SAKURA: On the Multi-hop Reasoning of Large Audio-Language Models Based on Speech and Audio Information
  8. 2026Multi-Source Evidence Fusion for Audio Question Answering