# Language Models as Few-Shot Learner for Task-Oriented Dialogue Systems

arXiv: https://arxiv.org/abs/2008.06239 · HTML: https://mlsanity.com/paper/2008.06239/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Aug 2020.

## Built on by (most cited first, 30 of 41)

- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization](https://mlsanity.com/paper/2406.01171/) (2024)
- [A Survey on Recent Advances in LLM-Based Multi-turn Dialogue Systems](https://mlsanity.com/paper/2402.18013/) (2024)
- [Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Bases](https://mlsanity.com/paper/2106.09231/) (2021)
- [Language Models are Few-shot Multilingual Learners](https://mlsanity.com/paper/2109.07684/) (2021)
- [Template-free Prompt Tuning for Few-shot NER](https://mlsanity.com/paper/2109.13532/) (2021)
- [Few-Shot Bot: Prompt-Based Learning for Dialogue Systems](https://mlsanity.com/paper/2110.08118/) (2021)
- [Towards Few-Shot Fact-Checking via Perplexity](https://mlsanity.com/paper/2103.09535/) (2021)
- [Leveraging Slot Descriptions for Zero-Shot Cross-Domain Dialogue State Tracking](https://mlsanity.com/paper/2105.04222/) (2021)
- [Are LLMs All You Need for Task-Oriented Dialogue?](https://mlsanity.com/paper/2304.06556/) (2023)
- [Description-Driven Task-Oriented Dialog Modeling](https://mlsanity.com/paper/2201.08904/) (2022)
- [Hallucination Detection: Robustly Discerning Reliable Answers in Large Language Models](https://mlsanity.com/paper/2407.04121/) (2024)
- [Communicating Natural Programs to Humans and Machines](https://mlsanity.com/paper/2106.07824/) (2021)
- [Reliable LLM-based User Simulator for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2402.13374/) (2024)
- [Multi-Stage Prompting for Knowledgeable Dialogue Generation](https://mlsanity.com/paper/2203.08745/) (2022)
- [An Overview on Language Models: Recent Developments and Outlook](https://mlsanity.com/paper/2303.05759/) (2023)
- [In-Context Learning User Simulators for Task-Oriented Dialog Systems](https://mlsanity.com/paper/2306.00774/) (2023)
- [Show, Don't Tell: Demonstrations Outperform Descriptions for Schema-Guided Task-Oriented Dialogue](https://mlsanity.com/paper/2204.04327/) (2022)
- [CrossCodeBench: Benchmarking Cross-Task Generalization of Source Code Models](https://mlsanity.com/paper/2302.04030/) (2023)
- [Enabling Classifiers to Make Judgements Explicitly Aligned with Human Values](https://mlsanity.com/paper/2210.07652/) (2022)
- [Farmer.Chat: Scaling AI-Powered Agricultural Services for Smallholder Farmers](https://mlsanity.com/paper/2409.08916/) (2024)
- [PatFig: Generating Short and Long Captions for Patent Figures](https://mlsanity.com/paper/2309.08379/) (2023)
- [Long-Horizon Dialogue Understanding for Role Identification in the Game of Avalon with Large Language Models](https://mlsanity.com/paper/2311.05720/) (2023)
- [GenSF: Simultaneous Adaptation of Generative Pre-trained Models and Slot Filling](https://mlsanity.com/paper/2106.07055/) (2021)
- [Improved and Efficient Conversational Slot Labeling through Question Answering](https://mlsanity.com/paper/2204.02123/) (2022)
- [Molecule Joint Auto-Encoding: Trajectory Pretraining with 2D and 3D Diffusion](https://mlsanity.com/paper/2312.03475/) (2023)
- [Metaphorical User Simulators for Evaluating Task-oriented Dialogue Systems](https://mlsanity.com/paper/2204.00763/) (2022)
- [Enhancing Large Language Model Induced Task-Oriented Dialogue Systems Through Look-Forward Motivated Goals](https://mlsanity.com/paper/2309.08949/) (2023)
- [Prompter: Zero-shot Adaptive Prefixes for Dialogue State Tracking Domain Adaptation](https://mlsanity.com/paper/2306.04724/) (2023)
- [N-Shot Learning for Augmenting Task-Oriented Dialogue State Tracking](https://mlsanity.com/paper/2103.00293/) (2021)

## Abstract (arXiv)

Task-oriented dialogue systems use four connected modules, namely, Natural Language Understanding (NLU), a Dialogue State Tracking (DST), Dialogue Policy (DP) and Natural Language Generation (NLG). A research challenge is to learn each module with the least amount of samples (i.e., few-shots) given the high cost related to the data collection. The most common and effective technique to solve this problem is transfer learning, where large language models, either pre-trained on text or task-specific data, are fine-tuned on the few samples. These methods require fine-tuning steps and a set of parameters for each task. Differently, language models, such as GPT-2 (Radford et al., 2019) and GPT-3 (Brown et al., 2020), allow few-shot learning by priming the model with few examples. In this paper, we evaluate the priming few-shot ability of language models in the NLU, DST, DP and NLG tasks. Importantly, we highlight the current limitations of this approach, and we discuss the possible implication for future work.