# Retrieve-and-Fill for Scenario-based Task-Oriented Semantic Parsing

arXiv: https://arxiv.org/abs/2202.00901 · HTML: https://mlsanity.com/paper/2202.00901/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 42 references that are in the map.
- First posted on arXiv 2 Feb 2022.

## Builds on (most cited first, 30 of 36)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Improving language models by retrieving from trillions of tokens](https://mlsanity.com/paper/2112.04426/) (2021)
- [Making Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2012.15723/) (2020)
- [Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval](https://mlsanity.com/paper/2007.00808/) (2020)
- [It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://mlsanity.com/paper/2009.07118/) (2020)
- [Regularizing Neural Networks by Penalizing Confident Output Distributions](https://mlsanity.com/paper/1701.06548/) (2017)
- [Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces](https://mlsanity.com/paper/1805.10190/) (2018)
- [Good-Enough Compositional Data Augmentation](https://mlsanity.com/paper/1904.09545/) (2019)
- [MTOP: A Comprehensive Multilingual Task-Oriented Semantic Parsing Benchmark](https://mlsanity.com/paper/2008.09335/) (2020)
- [Better Fine-Tuning by Reducing Representational Collapse](https://mlsanity.com/paper/2008.03156/) (2020)
- [Optimizing Dense Retrieval Model Training with Hard Negatives](https://mlsanity.com/paper/2104.08051/) (2021)
- [Coarse-to-Fine Decoding for Neural Semantic Parsing](https://mlsanity.com/paper/1805.04793/) (2018)
- [Semantic Parsing for Task Oriented Dialog using Hierarchical Representations](https://mlsanity.com/paper/1810.07942/) (2018)
- [SmBoP: Semi-autoregressive Bottom-up Semantic Parsing](https://mlsanity.com/paper/2010.12412/) (2020)
- [Low-Resource Domain Adaptation for Compositional Task-Oriented Semantic Parsing](https://mlsanity.com/paper/2010.03546/) (2020)
- [Neural Machine Translation with Monolingual Translation Memory](https://mlsanity.com/paper/2105.11269/) (2021)
- [Don't Parse, Generate! A Sequence to Sequence Architecture for Task-Oriented Semantic Parsing](https://mlsanity.com/paper/2001.11458/) (2020)
- [Domain-matched Pre-training Tasks for Dense Retrieval](https://mlsanity.com/paper/2107.13602/) (2021)
- [Improving Compositional Generalization in Semantic Parsing](https://mlsanity.com/paper/2010.05647/) (2020)
- [Conversational Semantic Parsing](https://mlsanity.com/paper/2009.13655/) (2020)
- [Controllable Semantic Parsing via Retrieval Augmentation](https://mlsanity.com/paper/2110.08458/) (2021)
- [Span Pointer Networks for Non-Autoregressive Task-Oriented Semantic Parsing](https://mlsanity.com/paper/2104.07275/) (2021)
- [Non-Autoregressive Semantic Parsing for Compositional Task-Oriented Dialog](https://mlsanity.com/paper/2104.04923/) (2021)
- [Translate & Fill: Improving Zero-Shot Multilingual Semantic Parsing with Synthetic Data](https://mlsanity.com/paper/2109.04319/) (2021)

## Built on by (most cited first, 7 of 7)

- [Generate-and-Retrieve: use your predictions to improve retrieval for semantic parsing](https://mlsanity.com/paper/2209.14899/) (2022)
- [TreePiece: Faster Semantic Parsing via Tree Tokenization](https://mlsanity.com/paper/2303.17161/) (2023)
- [Towards Zero-Shot Frame Semantic Parsing with Task Agnostic Ontologies and Simple Labels](https://mlsanity.com/paper/2305.03793/) (2023)
- [Low-Resource Compositional Semantic Parsing with Concept Pretraining](https://mlsanity.com/paper/2301.09809/) (2023)
- [Shift-Reduce Task-Oriented Semantic Parsing with Stack-Transformers](https://mlsanity.com/paper/2210.11984/) (2022)
- [PRoDeliberation: Parallel Robust Deliberation for End-to-End Spoken Language Understanding](https://mlsanity.com/paper/2406.07823/) (2024)
- [Adaptive Data Flywheel: Applying MAPE Control Loops to AI Agent Improvement](https://mlsanity.com/paper/2510.27051/) (2025)

## Abstract (arXiv)

Task-oriented semantic parsing models have achieved strong results in recent years, but unfortunately do not strike an appealing balance between model size, runtime latency, and cross-domain generalizability. We tackle this problem by introducing scenario-based semantic parsing: a variant of the original task which first requires disambiguating an utterance's "scenario" (an intent-slot template with variable leaf spans) before generating its frame, complete with ontology and utterance tokens. This formulation enables us to isolate coarse-grained and fine-grained aspects of the task, each of which we solve with off-the-shelf neural modules, also optimizing for the axes outlined above. Concretely, we create a Retrieve-and-Fill (RAF) architecture comprised of (1) a retrieval module which ranks the best scenario given an utterance and (2) a filling module which imputes spans into the scenario to create the frame. Our model is modular, differentiable, interpretable, and allows us to garner extra supervision from scenarios. RAF achieves strong results in high-resource, low-resource, and multilingual settings, outperforming recent approaches by wide margins despite, using base pre-trained encoders, small sequence lengths, and parallel decoding.