# Spoken Language Understanding on Unseen Tasks With In-Context Learning

arXiv: https://arxiv.org/abs/2505.07731 · HTML: https://mlsanity.com/paper/2505.07731/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 36 references that are in the map.
- First posted on arXiv 12 May 2025.

## Builds on (most cited first, 30 of 32)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks](https://mlsanity.com/paper/2204.07705/) (2022)
- [SALMONN: Towards Generic Hearing Abilities for Large Language Models](https://mlsanity.com/paper/2310.13289/) (2023)
- [MetaICL: Learning to Learn In Context](https://mlsanity.com/paper/2110.15943/) (2021)
- [Speech Model Pre-training for End-to-End Spoken Language Understanding](https://mlsanity.com/paper/1904.03670/) (2019)
- [Mitigating Label Biases for In-context Learning](https://mlsanity.com/paper/2305.19148/) (2023)
- [UPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluation](https://mlsanity.com/paper/2303.08518/) (2023)
- [Symbol tuning improves in-context learning in language models](https://mlsanity.com/paper/2305.08298/) (2023)
- [SLUE: New Benchmark Tasks for Spoken Language Understanding Evaluation on Natural Speech](https://mlsanity.com/paper/2111.10367/) (2021)
- [In-context Learning with Retrieved Demonstrations for Language Models: A Survey](https://mlsanity.com/paper/2401.11624/) (2024)
- [Many-Shot In-Context Learning in Multimodal Foundation Models](https://mlsanity.com/paper/2405.09798/) (2024)
- [Earnings-21: A Practical Benchmark for ASR in the Wild](https://mlsanity.com/paper/2104.11348/) (2021)
- [ESPnet-SLU: Advancing Spoken Language Understanding through ESPnet](https://mlsanity.com/paper/2111.14706/) (2021)
- [Visual Instruction Tuning with Polite Flamingo](https://mlsanity.com/paper/2307.01003/) (2023)
- [SLUE Phase-2: A Benchmark Suite of Diverse Spoken Language Understanding Tasks](https://mlsanity.com/paper/2212.10525/) (2022)
- [Can Whisper perform speech-based in-context learning?](https://mlsanity.com/paper/2309.07081/) (2023)
- [COSMIC: Data Efficient Instruction-tuning For Speech In-Context Learning](https://mlsanity.com/paper/2311.02248/) (2023)
- [Instruction-following Evaluation through Verbalizer Manipulation](https://mlsanity.com/paper/2307.10558/) (2023)
- [Towards Multimodal In-Context Learning for Vision & Language Models](https://mlsanity.com/paper/2403.12736/) (2024)
- [Speech Sentiment Analysis via Pre-trained Features from End-to-end ASR Models](https://mlsanity.com/paper/1911.09762/) (2019)
- [Can ChatGPT Detect Intent? Evaluating Large Language Models for Spoken Language Understanding](https://mlsanity.com/paper/2305.13512/) (2023)
- [A Study on the Integration of Pre-trained SSL, ASR, LM and SLU Models for Spoken Language Understanding](https://mlsanity.com/paper/2211.05869/) (2022)
- [End-to-End Spoken Language Understanding for Generalized Voice Assistants](https://mlsanity.com/paper/2106.09009/) (2021)
- [BGE Landmark Embedding: A Chunking-Free Embedding Method For Retrieval Augmented Long-Context Large Language Models](https://mlsanity.com/paper/2402.11573/) (2024)
- [Language Models can Exploit Cross-Task In-context Learning for Data-Scarce Novel Tasks](https://mlsanity.com/paper/2405.10548/) (2024)
- [Analyzing and Adapting Large Language Models for Few-Shot Multilingual NLU: Are We There Yet?](https://mlsanity.com/paper/2403.01929/) (2024)
- [Lexico-acoustic Neural-based Models for Dialog Act Classification](https://mlsanity.com/paper/1803.00831/) (2018)

## Built on by (most cited first, 2 of 2)

- [Exploring Fine-Tuning of Large Audio Language Models for Spoken Language Understanding under Limited Speech Data](https://mlsanity.com/paper/2509.15389/) (2025)
- [In-Context Adaptation of Encoder-Decoder Models in Speech Recognition](https://mlsanity.com/paper/2609.33865/) (2026)

## Abstract (arXiv)

Spoken language understanding (SLU) tasks involve diverse skills that probe the information extraction, classification and/or generation capabilities of models. In this setting, task-specific training data may not always be available. While traditional task-specific SLU models are unable to cater to such requirements, the speech-text large language models (LLMs) offer a promising alternative with emergent abilities. However, out of-the-box, our evaluations indicate that the zero/few-shot performance of prominent open-source speech-text LLMs on SLU tasks are not up to the mark. In this paper, we introduce a novel approach to robust task-agnostic fine-tuning using randomized class labels. With this proposed fine-tuning, we illustrate that the performance of the speech-text LLMs on an unseen task is significantly improved over standard approaches. Critically, the proposed approach avoids the requirement of task-specific data annotations for enabling new tasks in speech-text LLMs.