Zero-Shot Spoken Language Understanding with SLAM-ASR: What Do Monolingual ASR-Trained Linear Projectors Enable?
On the map: Speech and audio language models → Speech recognition models and training also fits: Multilingual and low-resource ASR
Abstract (arXiv)
The SLAM-ASR framework connects the features of a frozen speech encoder as input to a frozen LLM decoder to enable speech transcription. In this work we ask whether this training-on speech transcription-provides the LLM with the capability to also perform spoken language understanding (SLU) tasks, by training monolingual projectors on 11 languages. When prompted to answer a yes/no question about the input speech, the mean instruction following rate (IFR) is 49.8%. When not following the instruction, these models default back to transcription. The transcription quality-measured by the character error rate (CER)-is not sufficient to predict IFR. Remarkably, when the model is instructed to answer the topic question as a single digit option, the mean IFR rises to 81.3%. Our models score a mean 23.2% topic balanced accuracy (BAcc) against 14.3% chance. However, the same decoder scores a mean 71.5% BAcc when using reference transcripts as text input. This indicates a failure of the projector to transfer information for the SLU tasks.
Key facts. No citing papers in the map yet (data through 19 Oct 2026). Placed in Speech recognition models and training, one of 11 directions in Speech and audio language models. Builds on 9 of its 20 references that are in the map. First posted on arXiv 14 Oct 2026.
Builds on
9 of 20 references.
Closest ancestors · 5
Same direction, newest first
- 2026Reducing Prompt Sensitivity in LLM-based Speech Recognition Through Learnable Projection28 Jan
- 2024AlignFormer: Modality Matching Can Achieve Better Zero-shot Instruction-Following Speech-LLM2 Dec
- 2024Performance evaluation of SLAM-ASR: The Good, the Bad, the Ugly, and the Way Forward6 Nov
- 2024An Embarrassingly Simple Approach for LLM with Strong ASR Capacity13 Feb
- 2022Robust Speech Recognition via Large-Scale Weak Supervision6 Dec
Other references · 4
- 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 539
- 2025Closing the Gap Between Text and Speech Understanding in LLMscited by 21
- 2025Speech LLMs in Low-Resource Scenarios: Data Volume Requirements and the Impact of Pretraining on High-Resource Languagescited by 9
- 2026Language-Aware Distillation for Multilingual Instruction-Following Speech LLMs with ASR-Only Supervisioncited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2021N-Best ASR Transformer: Enhancing SLU Performance using Multiple ASR Hypotheses
- 2023Leveraging Large Language Models for Exploiting ASR Uncertainty
- 2026MEUSLI: a Multilingual Projector for LLM-based ASR and Beyond
- 2026Dynamic Multi-Expert Projectors with Stabilized Routing for Multilingual Speech Recognition
- 2022ASR-Generated Text for Language Model Pre-training Applied to Speech Tasks
- 2023Modality Confidence Aware Training for Robust End-to-End Spoken Language Understanding
- 2021Do We Still Need Automatic Speech Recognition for Spoken Language Understanding?
- 2026SpeechMapper: Speech-to-text Embedding Projector for LLMs