mlsanityData to 7 Oct 2026

RETRONLU: Retrieval Augmented Task-Oriented Semantic Parsing

arXiv 2109.10410 ↗First posted 21 Sep 2021cs.CL cs.IR cs.LGcited by 15

On the map: Retrieval-augmented generation → Generation and context integration

Abstract (arXiv)

While large pre-trained language models accumulate a lot of knowledge in their parameters, it has been demonstrated that augmenting it with non-parametric retrieval-based memory has a number of benefits from accuracy improvements to data efficiency for knowledge-focused tasks, such as question answering. In this paper, we are applying retrieval-based modeling ideas to the problem of multi-domain task-oriented semantic parsing for conversational assistants. Our approach, RetroNLU, extends a sequence-to-sequence model architecture with a retrieval component, used to fetch existing similar examples and provide them as an additional input to the model. In particular, we analyze two settings, where we augment an input with (a) retrieved nearest neighbor utterances (utterance-nn), and (b) ground-truth semantic parses of nearest neighbor utterances (semparse-nn). Our technique outperforms the baseline method by 1.5% absolute macro-F1, especially at the low resource setting, matching the baseline model accuracy with only 40% of the data. Furthermore, we analyze the nearest neighbor retrieval component's quality, model sensitivity and break down the performance for semantic parses of different utterance complexity.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation. Builds on 35 of its 50 references that are in the map. First posted on arXiv 21 Sep 2021.

Builds on

35 of 50 references.

Closest ancestors · 2

Same direction, newest first

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks22 May
  2. 2020REALM: Retrieval-Augmented Language Model Pre-Training10 Feb

Other references · 33

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  5. 2017Billion-scale similarity search with GPUscited by 2,477
  6. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  7. 2019Machine Unlearningcited by 1,020
  8. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  9. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  10. 2014Memory Networkscited by 354
  11. 2020Nearest Neighbor Machine Translationcited by 209
  12. 2018Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learningcited by 203
21 more
  1. 2020MTOP: A Comprehensive Multilingual Task-Oriented Semantic Parsing Benchmarkcited by 156
  2. 2018Semantic Parsing for Task Oriented Dialog using Hierarchical Representationscited by 101
  3. 2020Pre-training via Paraphrasingcited by 92
  4. 2020DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented Dialoguecited by 89
  5. 2018Retrieve and Refine: Improved Sequence Generation Models For Dialoguecited by 82
  6. 2020Low-Resource Domain Adaptation for Compositional Task-Oriented Semantic Parsingcited by 72
  7. 2020Don't Parse, Generate! A Sequence to Sequence Architecture for Task-Oriented Semantic Parsingcited by 59
  8. 2020Cross-lingual Retrieval for Iterative Self-Supervised Trainingcited by 54
  9. 2020Augmenting Transformers with KNN-Based Composite Memory for Dialoguecited by 53
  10. 2020Recent Neural Methods on Slot Filling and Intent Classification for Task-Oriented Dialogue Systems: A Surveycited by 51
  11. 2018Improving Natural Language Inference Using External Knowledge in the Science Questions Domaincited by 48
  12. 2017Neural Natural Language Inference Models Enhanced with External Knowledgecited by 40
  13. 2020Conversational Semantic Parsing for Dialog State Trackingcited by 40
  14. 2020Conversational Semantic Parsingcited by 39
  15. 2019Non-Parametric Adaptation for Neural Machine Translationcited by 34
  16. 2021Incorporating External Knowledge to Enhance Tabular Reasoningcited by 23
  17. 2021Span Pointer Networks for Non-Autoregressive Task-Oriented Semantic Parsingcited by 18
  18. 2019Multilingual Graphemic Hybrid ASR with Massive Data Augmentationcited by 13
  19. 2020Multilingual Neural RST Discourse Parsingcited by 13
  20. 2018Knowledge-Aware Conversational Semantic Parsing Over Web Tablescited by 6
  21. 2021Diagnosing Transformers in Task-Oriented Semantic Parsingcited by 5

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 7; 2023: 4; 2024: 1; 2025: 2.

15 papers.

Sort:

By area:

  1. 2022Diverse Demonstrations Improve In-context Compositional Generalizationcited by 136
  2. 2021Controllable Semantic Parsing via Retrieval Augmentationcited by 35
  3. 2022You can't pick your neighbors, or can you? When and how to rely on retrieval in the kNN-LMcited by 18
  4. 2023Towards Knowledge-Intensive Text-to-SQL Semantic Parsing with Formulaic Knowledgecited by 18
  5. 2022Generate-and-Retrieve: use your predictions to improve retrieval for semantic parsingcited by 15
  6. 2022Retrieve-and-Fill for Scenario-based Task-Oriented Semantic Parsingcited by 7
  7. 2022Diverse Parallel Data Synthesis for Cross-Database Adaptation of Text-to-SQL Parserscited by 7
  8. 2023Measuring and Mitigating Constraint Violations of In-Context Learning for Utterance-to-API Semantic Parsingcited by 6
  9. 2022Unveiling the Black Box of PLMs with Semantic Anchors: Towards Interpretable Neural Semantic Parsingcited by 5
  10. 2024Automatic Combination of Sample Selection Strategies for Few-Shot Learningcited by 4
  11. 2023Structured Case-based Reasoning for Inference-time Adaptation of Text-to-SQL parserscited by 3
  12. 2022DocuT5: Seq2seq SQL Generation with Table Documentationcited by 1
3 more
  1. 2023Evaluating Inter-Bilingual Semantic Parsing for Indian Languagescited by 1
  2. 2025Large Language Models are Demonstration Pre-Selectors for Themselvescited by 1
  3. 2025Handling Open-Vocabulary Constructs in Formalizing Specifications: Retrieval-Augmented Parsing with Expert Knowledgecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Retrieval-Augmented Semantic Parsing: Improving Generalization with Lexical Knowledge
  2. 2023BRENT: Bidirectional Retrieval Enhanced Norwegian Transformer
  3. 2024RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processing
  4. 2023Surface-Based Retrieval Reduces Perplexity of Retrieval-Augmented Language Models
  5. 2022kNN-Prompt: Nearest Neighbor Zero-Shot Inference
  6. 2024Reliable, Adaptable, and Attributable Language Models with Retrieval
  7. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Context
  8. 2021Improving language models by retrieving from trillions of tokens