mlsanityData to 14 Oct 2026

Exploring Fine-Tuning of Large Audio Language Models for Spoken Language Understanding under Limited Speech Data

arXiv 2509.15389 ↗First posted 18 Sep 2025cs.SD cs.CL cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Large Audio Language Models (LALMs) have emerged as powerful tools for speech-related tasks but remain underexplored for fine-tuning, especially with limited speech data. To bridge this gap, we systematically examine how different fine-tuning schemes including text-only, direct mixing, and curriculum learning affect spoken language understanding (SLU), focusing on scenarios where text-label pairs are abundant while paired speech-label data are limited. Results show that LALMs already achieve competitive performance with text-only fine-tuning, highlighting their strong generalization ability. Adding even small amounts of speech data (2-5%) yields substantial further gains, with curriculum learning particularly effective under scarce data. In cross-lingual SLU, combining source-language speech data with target-language text and minimal target-language speech data enables effective adaptation. Overall, this study provides practical insights into the LALM fine-tuning under realistic data constraints.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 21 of its 22 references that are in the map. First posted on arXiv 18 Sep 2025.

Builds on

21 of 22 references.

References · 21

  1. 2023Instruction Tuning for Large Language Models: A Surveycited by 623
  2. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
  3. 2024Qwen2-Audio Technical Reportcited by 560
  4. 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 506
  5. 2023AudioPaLM: A Large Language Model That Can Speak and Listencited by 343
  6. 2020SLURP: A Spoken Language Understanding Resource Packagecited by 198
  7. 2022MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languagescited by 146
  8. 2023AudioChatLlama: Towards General-Purpose Speech Abilities for LLMscited by 59
  9. 2021A Survey on Spoken Language Understanding: Recent Advances and New Frontierscited by 53
  10. 2025Omni-R1: Do You Really Need Audio to Fine-Tune Your Audio LLM?cited by 46
  11. 2020Leveraging Unpaired Text Data for Training End-to-End Speech-to-Intent Systemscited by 38
  12. 2022Towards Reducing the Need for Speech Training Data To Build Spoken Language Understanding Systemscited by 20
9 more
  1. 2024Speech-MASSIVE: A Multilingual Speech Dataset for SLU and Beyondcited by 20
  2. 2024Just ASR + LLM? A Study on Speech Large Language Models' Ability to Identify and Understand Speaker in Spoken Dialoguecited by 16
  3. 2023ITALIC: An Italian Intent Classification Datasetcited by 10
  4. 2025Exploring Finetuned Audio-LLM on Heart Murmur Featurescited by 5
  5. 2024An End-to-End Speech Summarization Using Large Language Modelcited by 4
  6. 2025Direct Speech to Speech Translation: A Reviewcited by 4
  7. 2023Leveraging Pretrained ASR Encoders for Effective and Efficient End-to-End Speech Intent Classification and Slot Fillingcited by 3
  8. 2025Fine-Tuning Large Audio-Language Models with LoRA for Precise Temporal Localization of Prolonged Exposure Therapy Elementscited by 3
  9. 2025Spoken Language Understanding on Unseen Tasks With In-Context Learningcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026Benign Fine-Tuning Breaks Safety Alignment in Audio LLMscited by 1
  2. 2026Analyzing Error Propagation in Korean Spoken QA with ASR-LLM Cascadescited by 1
  3. 2026CALM: Class-Conditional Sparse Attention Vectors for Large Audio-Language Modelscited by 0
  4. 2026CORTIS: Text-Only Adaptation of Spoken Language Models for Task-Oriented Voice Agentscited by 0
  5. 2026Efficiently Adapting Spoken Language Models for the Singaporean Contextcited by 0
  6. 2026Robust Summarization of Doctor-Patient Conversations: TalTech Systems for the Beyond Transcription Challengecited by 0