# A Few-Shot Semantic Parser for Wizard-of-Oz Dialogues with the Precise ThingTalk Representation

arXiv: https://arxiv.org/abs/2009.07968 · HTML: https://mlsanity.com/paper/2009.07968/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Sep 2020.

## Built on by (most cited first, 8 of 8)

- [Prompting Large Language Models With the Socratic Method](https://mlsanity.com/paper/2303.08769/) (2023)
- [SPINACH: SPARQL-Based Information Navigation for Challenging Real-World Questions](https://mlsanity.com/paper/2407.11417/) (2024)
- ["Do you follow me?": A Survey of Recent Approaches in Dialogue State Tracking](https://mlsanity.com/paper/2207.14627/) (2022)
- [SUQL: Conversational Search over Structured and Unstructured Data with Large Language Models](https://mlsanity.com/paper/2311.09818/) (2023)
- [ThingTalk: An Extensible, Executable Representation Language for Task-Oriented Dialogues](https://mlsanity.com/paper/2203.12751/) (2022)
- [Dialogue Meaning Representation for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2204.10989/) (2022)
- [Can Current Task-oriented Dialogue Models Automate Real-world Scenarios in the Wild?](https://mlsanity.com/paper/2212.10504/) (2022)
- [Benchmarks Underestimate the Readiness of Multi-lingual Dialogue Agents](https://mlsanity.com/paper/2405.17840/) (2024)

## Abstract (arXiv)

Previous attempts to build effective semantic parsers for Wizard-of-Oz (WOZ) conversations suffer from the difficulty in acquiring a high-quality, manually annotated training set. Approaches based only on dialogue synthesis are insufficient, as dialogues generated from state-machine based models are poor approximations of real-life conversations. Furthermore, previously proposed dialogue state representations are ambiguous and lack the precision necessary for building an effective agent. This paper proposes a new dialogue representation and a sample-efficient methodology that can predict precise dialogue states in WOZ conversations. We extended the ThingTalk representation to capture all information an agent needs to respond properly. Our training strategy is sample-efficient: we combine (1) fewshot data sparsely sampling the full dialogue space and (2) synthesized data covering a subset space of dialogues generated by a succinct state-based dialogue model. The completeness of the extended ThingTalk language is demonstrated with a fully operational agent, which is also used in training data synthesis. We demonstrate the effectiveness of our methodology on MultiWOZ 3.0, a reannotation of the MultiWOZ 2.1 dataset in ThingTalk. ThingTalk can represent 98% of the test turns, while the simulator can emulate 85% of the validation set. We train a contextual semantic parser using our strategy, and obtain 79% turn-by-turn exact match accuracy on the reannotated test set.