# Low-Resource Task-Oriented Semantic Parsing via Intrinsic Modeling

arXiv: https://arxiv.org/abs/2104.07224 · HTML: https://mlsanity.com/paper/2104.07224/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 21 references that are in the map.
- First posted on arXiv 15 Apr 2021.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data](https://mlsanity.com/paper/2005.08314/) (2020)
- [MTOP: A Comprehensive Multilingual Task-Oriented Semantic Parsing Benchmark](https://mlsanity.com/paper/2008.09335/) (2020)
- [Data Recombination for Neural Semantic Parsing](https://mlsanity.com/paper/1606.03622/) (2016)
- [Semantic Parsing for Task Oriented Dialog using Hierarchical Representations](https://mlsanity.com/paper/1810.07942/) (2018)
- [Low-Resource Domain Adaptation for Compositional Task-Oriented Semantic Parsing](https://mlsanity.com/paper/2010.03546/) (2020)
- [Towards Zero-Shot Frame Semantic Parsing for Domain Scaling](https://mlsanity.com/paper/1707.02363/) (2017)
- [Conversational Semantic Parsing](https://mlsanity.com/paper/2009.13655/) (2020)
- [Robust Zero-Shot Cross-Domain Slot Filling with Example Values](https://mlsanity.com/paper/1906.06870/) (2019)
- [Don't paraphrase, detect! Rapid and Effective Data Collection for Semantic Parsing](https://mlsanity.com/paper/1908.09940/) (2019)
- [Zero-Shot Adaptive Transfer for Conversational Language Understanding](https://mlsanity.com/paper/1808.10059/) (2018)
- [Improving Semantic Parsing for Task Oriented Dialog](https://mlsanity.com/paper/1902.06000/) (2019)

## Built on by (most cited first, 10 of 10)

- [Constrained Language Models Yield Few-Shot Semantic Parsers](https://mlsanity.com/paper/2104.08768/) (2021)
- [TAGPRIME: A Unified Framework for Relational Structure Extraction](https://mlsanity.com/paper/2205.12585/) (2022)
- [Compositional Task-Oriented Parsing as Abstractive Question Answering](https://mlsanity.com/paper/2205.02068/) (2022)
- [Semantic Parsing in Task-Oriented Dialog with Recursive Insertion-based Encoder](https://mlsanity.com/paper/2109.04500/) (2021)
- [Retrieve-and-Fill for Scenario-based Task-Oriented Semantic Parsing](https://mlsanity.com/paper/2202.00901/) (2022)
- [kNN-ICL: Compositional Task-Oriented Parsing Generalization with Nearest Neighbor In-Context Learning](https://mlsanity.com/paper/2312.10771/) (2023)
- [Towards Zero-Shot Frame Semantic Parsing with Task Agnostic Ontologies and Simple Labels](https://mlsanity.com/paper/2305.03793/) (2023)
- [Transfer Learning for Multi-lingual Tasks -- a Survey](https://mlsanity.com/paper/2110.02052/) (2021)
- [Low-Resource Compositional Semantic Parsing with Concept Pretraining](https://mlsanity.com/paper/2301.09809/) (2023)
- [Assessing Data Efficiency in Task-Oriented Semantic Parsing](https://mlsanity.com/paper/2107.04736/) (2021)

## Abstract (arXiv)

Task-oriented semantic parsing models typically have high resource requirements: to support new ontologies (i.e., intents and slots), practitioners crowdsource thousands of samples for supervised fine-tuning. Partly, this is due to the structure of de facto copy-generate parsers; these models treat ontology labels as discrete entities, relying on parallel data to extrinsically derive their meaning. In our work, we instead exploit what we intrinsically know about ontology labels; for example, the fact that SL:TIME_ZONE has the categorical type "slot" and language-based span "time zone". Using this motivation, we build our approach with offline and online stages. During preprocessing, for each ontology label, we extract its intrinsic properties into a component, and insert each component into an inventory as a cache of sorts. During training, we fine-tune a seq2seq, pre-trained transformer to map utterances and inventories to frames, parse trees comprised of utterance and ontology tokens. Our formulation encourages the model to consider ontology labels as a union of its intrinsic properties, therefore substantially bootstrapping learning in low-resource settings. Experiments show our model is highly sample efficient: using a low-resource benchmark derived from TOPv2, our inventory parser outperforms a copy-generate parser by +15 EM absolute (44% relative) when fine-tuning on 10 samples from an unseen domain.