mlsanityData to 14 Oct 2026

Factors affecting the in-context learning abilities of LLMs for dialogue state tracking

arXiv 2506.08753 ↗First posted 10 Jun 2025cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

This study explores the application of in-context learning (ICL) to the dialogue state tracking (DST) problem and investigates the factors that influence its effectiveness. We use a sentence embedding based k-nearest neighbour method to retrieve the suitable demonstrations for ICL. The selected demonstrations, along with the test samples, are structured within a template as input to the LLM. We then conduct a systematic study to analyse the impact of factors related to demonstration selection and prompt context on DST performance. This work is conducted using the MultiWoZ2.4 dataset and focuses primarily on the OLMo-7B-instruct, Mistral-7B-Instruct-v0.3, and Llama3.2-3B-Instruct models. Our findings provide several useful insights on in-context learning abilities of LLMs for dialogue state tracking.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 24 references that are in the map. First posted on arXiv 10 Jun 2025.

Builds on

22 of 24 references.

References · 22

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  4. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  5. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  6. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  7. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 766
  8. 2021Learning To Retrieve Prompts for In-Context Learningcited by 648
  9. 2020Language-agnostic BERT Sentence Embeddingcited by 627
  10. 2024OLMo: Accelerating the Science of Language Modelscited by 613
  11. 2022Active Example Selection for In-Context Learningcited by 196
  12. 2023Unified Demonstration Retriever for In-Context Learningcited by 142
10 more
  1. 2023Finding Support Examples for In-Context Learningcited by 104
  2. 2021MultiWOZ 2.4: A Multi-Domain Task-Oriented Dialogue Dataset with Essential Annotation Corrections to Improve State Tracking Evaluationcited by 88
  3. 2022Data Curation Alone Can Stabilize In-context Learningcited by 59
  4. 2024Revisiting Demonstration Selection Strategies in In-Context Learningcited by 58
  5. 2023OpenICL: An Open-Source Framework for In-context Learningcited by 51
  6. 2022In-Context Learning for Few-Shot Dialogue State Trackingcited by 47
  7. 2022DiSTRICT: Dialogue State Tracking with Retriever Driven In-Context Tuningcited by 9
  8. 2024Dialog2Flow: Pre-training Soft-Contrastive Action-Driven Sentence Embeddings for Automatic Dialog Flow Extractioncited by 8
  9. 2023Stabilized In-Context Learning with Pre-trained Language Models for Few Shot Dialogue State Trackingcited by 6
  10. 2024Comparative Analysis of Demonstration Selection Algorithms for LLM In-Context Learningcited by 3

Built on by

Not cited yet.