mlsanityData to 13 Oct 2026

Improving Dialogue State Tracking through Combinatorial Search for In-Context Examples

arXiv 2506.00622 ↗First posted 31 May 2025cs.CL cs.AI cs.IRcited by 0

Not on the map. Why?

Abstract (arXiv)

In dialogue state tracking (DST), in-context learning comprises a retriever that selects labeled dialogues as in-context examples and a DST model that uses these examples to infer the dialogue state of the query dialogue. Existing methods for constructing training data for retrievers suffer from three key limitations: (1) the synergistic effect of examples is not considered, (2) the linguistic characteristics of the query are not sufficiently factored in, and (3) scoring is not directly optimized for DST performance. Consequently, the retriever can fail to retrieve examples that would substantially improve DST performance. To address these issues, we present CombiSearch, a method that scores effective in-context examples based on their combinatorial impact on DST performance. Our evaluation on MultiWOZ shows that retrievers trained with CombiSearch surpass state-of-the-art models, achieving a 20x gain in data efficiency and generalizing well to the SGD dataset. Moreover, CombiSearch attains a 12% absolute improvement in the upper bound DST performance over traditional approaches when no retrieval errors are assumed. This significantly increases the headroom for practical DST performance while demonstrating that existing methods rely on suboptimal data for retriever training.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 26 of its 39 references that are in the map. First posted on arXiv 31 May 2025.

Builds on

26 of 39 references.

References · 26

  1. 2020Language Models are Few-Shot Learnerscited by 32,320
  2. 2024The Llama 3 Herd of Modelscited by 14,486
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,979
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,107
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,712
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,317
  7. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,500
  8. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,191
  9. 2021Learning To Retrieve Prompts for In-Context Learningcited by 643
  10. 2022Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoningcited by 347
  11. 2020A Simple Language Model for Task-Oriented Dialoguecited by 331
  12. 2021Surface Form Competition: Why the Highest Probability Answer Isn't Always Rightcited by 261
14 more
  1. 2023Better Zero-Shot Reasoning with Role-Play Promptingcited by 221
  2. 2019Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systemscited by 216
  3. 2023Compositional Exemplars for In-context Learningcited by 144
  4. 2022Diverse Demonstrations Improve In-context Compositional Generalizationcited by 137
  5. 2024Does Prompt Formatting Have Any Impact on LLM Performance?cited by 135
  6. 2021Dialogue State Tracking with a Language Model using Schema-Driven Promptingcited by 94
  7. 2023In-context Example Selection with Influencescited by 60
  8. 2023Coverage-based Example Selection for In-Context Learningcited by 59
  9. 2022Data Curation Alone Can Stabilize In-context Learningcited by 58
  10. 2024Revisiting Demonstration Selection Strategies in In-Context Learningcited by 57
  11. 2020MultiWOZ 2.3: A multi-domain task-oriented dialogue dataset enhanced with annotation corrections and co-reference annotationcited by 51
  12. 2024SynthDST: Synthetic Data is All You Need for Few-Shot Dialog State Trackingcited by 17
  13. 2022DiSTRICT: Dialogue State Tracking with Retriever Driven In-Context Tuningcited by 9
  14. 2023Stabilized In-Context Learning with Pre-trained Language Models for Few Shot Dialogue State Trackingcited by 6

Built on by

Not cited yet.