# DiSTRICT: Dialogue State Tracking with Retriever Driven In-Context Tuning

arXiv: https://arxiv.org/abs/2212.02851 · HTML: https://mlsanity.com/paper/2212.02851/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 39 references that are in the map.
- First posted on arXiv 6 Dec 2022.

## Builds on (most cited first, 30 of 31)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning](https://mlsanity.com/paper/2205.05638/) (2022)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Dataset](https://mlsanity.com/paper/1909.05855/) (2019)
- [A Network-based End-to-End Trainable Task-oriented Dialogue System](https://mlsanity.com/paper/1604.04562/) (2016)
- [UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models](https://mlsanity.com/paper/2201.05966/) (2022)
- [Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/1905.08743/) (2019)
- [Meta-learning via Language Model In-context Tuning](https://mlsanity.com/paper/2110.07814/) (2021)
- [Dialogue State Tracking with a Language Model using Schema-Driven Prompting](https://mlsanity.com/paper/2109.07506/) (2021)
- [Few-Shot Bot: Prompt-Based Learning for Dialogue Systems](https://mlsanity.com/paper/2110.08118/) (2021)
- [Leveraging Slot Descriptions for Zero-Shot Cross-Domain Dialogue State Tracking](https://mlsanity.com/paper/2105.04222/) (2021)
- [Zero-Shot Dialogue State Tracking via Cross-Task Transfer](https://mlsanity.com/paper/2109.04655/) (2021)
- [From Machine Reading Comprehension to Dialogue State Tracking: Bridging the Gap](https://mlsanity.com/paper/2004.05827/) (2020)
- [Robust Zero-Shot Cross-Domain Slot Filling with Example Values](https://mlsanity.com/paper/1906.06870/) (2019)
- [Zero-shot Generalization in Dialog State Tracking through Generative Question Answering](https://mlsanity.com/paper/2101.08333/) (2021)
- [CINS: Comprehensive Instruction for Few-shot Learning in Task-oriented Dialog Systems](https://mlsanity.com/paper/2109.04645/) (2021)
- [MA-DST: Multi-Attention Based Scalable Dialog State Tracking](https://mlsanity.com/paper/2002.08898/) (2020)
- ["Do you follow me?": A Survey of Recent Approaches in Dialogue State Tracking](https://mlsanity.com/paper/2207.14627/) (2022)
- [Dialogue Summaries as Dialogue States (DS2), Template-Guided Summarization for Few-shot Dialogue State Tracking](https://mlsanity.com/paper/2203.01552/) (2022)
- [Improving Limited Labeled Dialogue State Tracking with Self-Supervision](https://mlsanity.com/paper/2010.13920/) (2020)
- [Show, Don't Tell: Demonstrations Outperform Descriptions for Schema-Guided Task-Oriented Dialogue](https://mlsanity.com/paper/2204.04327/) (2022)

## Built on by (most cited first, 9 of 9)

- [Diverse Retrieval-Augmented In-Context Learning for Dialogue State Tracking](https://mlsanity.com/paper/2307.01453/) (2023)
- [Interpretable and Robust Dialogue State Tracking via Natural Language Summarization with LLMs](https://mlsanity.com/paper/2503.08857/) (2025)
- [Schema Graph-Guided Prompt for Multi-Domain Dialogue State Tracking](https://mlsanity.com/paper/2311.06345/) (2023)
- [An Approach to Build Zero-Shot Slot-Filling System for Industry-Grade Conversational Assistants](https://mlsanity.com/paper/2406.08848/) (2024)
- [In-Context Examples Matter: Improving Emotion Recognition in Conversation with Instruction Tuning](https://mlsanity.com/paper/2508.11889/) (2025)
- [Beyond Ontology in Dialogue State Tracking for Goal-Oriented Chatbot](https://mlsanity.com/paper/2410.22767/) (2024)
- [Improving Dialogue State Tracking through Combinatorial Search for In-Context Examples](https://mlsanity.com/paper/2506.00622/) (2025)
- [Factors affecting the in-context learning abilities of LLMs for dialogue state tracking](https://mlsanity.com/paper/2506.08753/) (2025)
- [Beyond More Context: Retrieval Diversity Boosts Multi-Turn Intent Understanding](https://mlsanity.com/paper/2510.17940/) (2025)

## Abstract (arXiv)

Dialogue State Tracking (DST), a key component of task-oriented conversation systems, represents user intentions by determining the values of pre-defined slots in an ongoing dialogue. Existing approaches use hand-crafted templates and additional slot information to fine-tune and prompt large pre-trained language models and elicit slot values from the dialogue context. Significant manual effort and domain knowledge is required to design effective prompts, limiting the generalizability of these approaches to new domains and tasks. In this work, we propose DiSTRICT, a generalizable in-context tuning approach for DST that retrieves highly relevant training examples for a given dialogue to fine-tune the model without any hand-crafted templates. Experiments with the MultiWOZ benchmark datasets show that DiSTRICT outperforms existing approaches in various zero-shot and few-shot settings using a much smaller model, thereby providing an important advantage for real-world deployments that often have limited resource availability.