# Efficient Retrieval Augmented Generation from Unstructured Knowledge for Task-Oriented Dialog

arXiv: https://arxiv.org/abs/2102.04643 · HTML: https://mlsanity.com/paper/2102.04643/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Placed in Conversational and speech RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 22 of its 32 references that are in the map.
- First posted on arXiv 9 Feb 2021.

## Placed in

- Retrieval-augmented generation / [Conversational and speech RAG](https://mlsanity.com/rag/conversational-and-speech-rag/)

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [An Overview of Multi-Task Learning in Deep Neural Networks](https://mlsanity.com/paper/1706.05098/) (2017)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [Wizard of Wikipedia: Knowledge-Powered Conversational agents](https://mlsanity.com/paper/1811.01241/) (2018)
- [Multi-Task Deep Neural Networks for Natural Language Understanding](https://mlsanity.com/paper/1901.11504/) (2019)
- [A Knowledge-Grounded Neural Conversation Model](https://mlsanity.com/paper/1702.01932/) (2017)
- [Hello, It's GPT-2 -- How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/1907.05774/) (2019)
- [Knowledge-Grounded Dialogue Generation with Pre-trained Language Models](https://mlsanity.com/paper/2010.08824/) (2020)
- [Sequential Latent Knowledge Selection for Knowledge-Grounded Dialogue](https://mlsanity.com/paper/2002.07510/) (2020)
- [Beyond Domain APIs: Task-oriented Conversational Modeling with Unstructured Knowledge Access](https://mlsanity.com/paper/2006.03533/) (2020)
- [Towards Exploiting Background Knowledge for Building Conversation Systems](https://mlsanity.com/paper/1809.08205/) (2018)

## Built on by (most cited first, 30 of 33)

- [Retrieval Augmentation Reduces Hallucination in Conversation](https://mlsanity.com/paper/2104.07567/) (2021)
- [CBR-RAG: Case-Based Reasoning for Retrieval Augmented Generation in LLMs for Legal Question Answering](https://mlsanity.com/paper/2404.04302/) (2024)
- [LitLLM: A Toolkit for Scientific Literature Review](https://mlsanity.com/paper/2402.01788/) (2024)
- [RAGChecker: A Fine-grained Framework for Diagnosing Retrieval-Augmented Generation](https://mlsanity.com/paper/2408.08067/) (2024)
- [Fast Nearest Neighbor Machine Translation](https://mlsanity.com/paper/2105.14528/) (2021)
- [FairRAG: Fair Human Generation via Fair Retrieval Augmentation](https://mlsanity.com/paper/2403.19964/) (2024)
- [Reason first, then respond: Modular Generation for Knowledge-infused Dialogue](https://mlsanity.com/paper/2111.05204/) (2021)
- [GNN-LM: Language Modeling based on Global Contexts via GNN](https://mlsanity.com/paper/2110.08743/) (2021)
- [RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processing](https://mlsanity.com/paper/2404.19543/) (2024)
- [Relational Memory Augmented Language Models](https://mlsanity.com/paper/2201.09680/) (2022)
- [RetGen: A Joint framework for Retrieval and Grounded Text Generation Modeling](https://mlsanity.com/paper/2105.06597/) (2021)
- [Accelerating Retrieval-Augmented Generation](https://mlsanity.com/paper/2412.15246/) (2024)
- [Simple and Scalable Nearest Neighbor Machine Translation](https://mlsanity.com/paper/2302.12188/) (2023)
- [Retrieval-Enhanced Machine Learning: Synthesis and Opportunities](https://mlsanity.com/paper/2407.12982/) (2024)
- [Faster Nearest Neighbor Machine Translation](https://mlsanity.com/paper/2112.08152/) (2021)
- [AARGH! End-to-end Retrieval-Generation for Task-Oriented Dialog](https://mlsanity.com/paper/2209.03632/) (2022)
- [kNN-NER: Named Entity Recognition with Nearest Neighbor Search](https://mlsanity.com/paper/2203.17103/) (2022)
- [Adapting Document-Grounded Dialog Systems to Spoken Conversations using Data Augmentation and a Noisy Channel Model](https://mlsanity.com/paper/2112.08844/) (2021)
- [OPERA: Harmonizing Task-Oriented Dialogs and Information Seeking Experience](https://mlsanity.com/paper/2206.12449/) (2022)
- [Knowledge-grounded Dialog State Tracking](https://mlsanity.com/paper/2210.06656/) (2022)
- [Cascaded Span Extraction and Response Generation for Document-Grounded Dialog](https://mlsanity.com/paper/2106.07275/) (2021)
- [Task-oriented Document-Grounded Dialog Systems by HLTPR@RWTH for DSTC9 and DSTC10](https://mlsanity.com/paper/2304.07101/) (2023)
- [Revealing User Familiarity Bias in Task-Oriented Dialogue via Interactive Evaluation](https://mlsanity.com/paper/2305.13857/) (2023)
- [End-to-End Task-Oriented Dialog Modeling with Semi-Structured Knowledge Management](https://mlsanity.com/paper/2106.11796/) (2021)
- [Controllable Factuality in Document-Grounded Dialog Systems Using a Noisy Channel Model](https://mlsanity.com/paper/2210.17418/) (2022)
- [GNN-SL: Sequence Labeling Based on Nearest Examples via GNN](https://mlsanity.com/paper/2212.02017/) (2022)
- [How Does Knowledge Selection Help Retrieval Augmented Generation?](https://mlsanity.com/paper/2410.13258/) (2024)
- [Query Enhanced Knowledge-Intensive Conversation via Unsupervised Joint Modeling](https://mlsanity.com/paper/2212.09588/) (2022)
- [Roles of MLLMs in Visually Rich Document Retrieval for RAG: A Survey](https://mlsanity.com/paper/2601.03262/) (2025)
- [Nearest Neighbor Machine Translation is Meta-Optimizer on Output Projection Layer](https://mlsanity.com/paper/2305.13034/) (2023)

## Abstract (arXiv)

This paper summarizes our work on the first track of the ninth Dialog System Technology Challenge (DSTC 9), "Beyond Domain APIs: Task-oriented Conversational Modeling with Unstructured Knowledge Access". The goal of the task is to generate responses to user turns in a task-oriented dialog that require knowledge from unstructured documents. The task is divided into three subtasks: detection, selection and generation. In order to be compute efficient, we formulate the selection problem in terms of hierarchical classification steps. We achieve our best results with this model. Alternatively, we employ siamese sequence embedding models, referred to as Dense Knowledge Retrieval, to retrieve relevant documents. This method further reduces the computation time by a factor of more than 100x at the cost of degradation in R@1 of 5-6% compared to the first model. Then for either approach, we use Retrieval Augmented Generation to generate responses based on multiple selected snippets and we show how the method can be used to fine-tune trained embeddings.