mlsanityData to 7 Oct 2026

Efficiently Adapting Spoken Language Models for the Singaporean Context

arXiv 2607.10092 ↗First posted 11 Jul 2026cs.CL cs.AIcited by 0

On the map: LLM reasoning → Audio-language model reasoning

Abstract (arXiv)

Spoken language models (SLMs) unify speech perception and reasoning, but adapting them to sensitive domains is underexplored, especially when the original training data is inaccessible and the use case demands multilingual, spoken-query interaction. We adapt an open-source SLM to the Singaporean Home Team context across five speech tasks in Singapore's four official languages, combining LoRA fine-tuning, a surrogate text-QA dataset that guards against catastrophic forgetting, and a multi-task objective that adapts the CoBa reweighting scheme to speech. We also build HTD-multilingual-QA, a 504,853 sample multilingual QA dataset in text and spoken form. The resulting HT-Moonstone (5B) matches or outperforms SLMs up to 7x its size on most tasks, attains the best accent and gender recognition among all models evaluated, and loses under 2% of its original speech QA ability.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning. Builds on 9 of its 21 references that are in the map. First posted on arXiv 11 Jul 2026.

Builds on

9 of 21 references.

References · 9

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  3. 2025On The Landscape of Spoken Language Models: A Comprehensive Surveycited by 116
  4. 2024A Survey on Speech Large Language Models for Understandingcited by 71
  5. 2025Advancing Singlish Understanding: Bridging the Gap with Datasets and Multimodal Modelscited by 8
  6. 2025SeaLLMs-Audio: Large Audio-Language Models for Southeast Asiacited by 7
  7. 2025Exploring Fine-Tuning of Large Audio Language Models for Spoken Language Understanding under Limited Speech Datacited by 6
  8. 2025CantoASR: Prosody-Aware ASR-LALM Collaboration for Low-Resource Cantonesecited by 3
  9. 2026Direct Preference Optimization for English-Mandarin Code-Switching Speech Recognition in Audio LLMscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024MERaLiON-AudioLLM: Bridging Audio and Language with Large Language Models
  2. 2026SEA-SpeechBench: A Large-Scale Multitask Benchmark for Speech Understanding Across Southeast Asia
  3. 2023On the Calibration of Multilingual Question Answering LLMs
  4. 2024MERaLiON-SpeechEncoder: Towards a Speech Foundation Model for Singapore and Beyond
  5. 2025NativQA Framework: Enabling LLMs and VLMs with Native, Local, and Everyday Knowledge
  6. 2026SEABED: SouthEast Asian Benchmark for Evaluating Audio Reasoning
  7. 2025VoxEval: Benchmarking the Knowledge Understanding Capabilities of End-to-End Spoken Language Models
  8. 2024Sailor: Open Language Models for South-East Asia