mlsanityData to 7 Oct 2026

Efficient Retrieval Augmented Generation from Unstructured Knowledge for Task-Oriented Dialog

arXiv 2102.04643 ↗First posted 9 Feb 2021cs.CLcited by 33

On the map: Retrieval-augmented generation → Conversational and speech RAG

Abstract (arXiv)

This paper summarizes our work on the first track of the ninth Dialog System Technology Challenge (DSTC 9), "Beyond Domain APIs: Task-oriented Conversational Modeling with Unstructured Knowledge Access". The goal of the task is to generate responses to user turns in a task-oriented dialog that require knowledge from unstructured documents. The task is divided into three subtasks: detection, selection and generation. In order to be compute efficient, we formulate the selection problem in terms of hierarchical classification steps. We achieve our best results with this model. Alternatively, we employ siamese sequence embedding models, referred to as Dense Knowledge Retrieval, to retrieve relevant documents. This method further reduces the computation time by a factor of more than 100x at the cost of degradation in R@1 of 5-6% compared to the first model. Then for either approach, we use Retrieval Augmented Generation to generate responses based on multiple selected snippets and we show how the method can be used to fine-tune trained embeddings.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Placed in Conversational and speech RAG, one of 15 directions in Retrieval-augmented generation. Builds on 22 of its 32 references that are in the map. First posted on arXiv 9 Feb 2021.

Builds on

22 of 32 references.

References · 22

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  7. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  8. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  9. 2017Billion-scale similarity search with GPUscited by 2,477
  10. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  11. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  12. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 791
10 more
  1. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 760
  2. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
  3. 2018Wizard of Wikipedia: Knowledge-Powered Conversational agentscited by 530
  4. 2019Multi-Task Deep Neural Networks for Natural Language Understandingcited by 349
  5. 2017A Knowledge-Grounded Neural Conversation Modelcited by 136
  6. 2019Hello, It's GPT-2 -- How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systemscited by 110
  7. 2020Knowledge-Grounded Dialogue Generation with Pre-trained Language Modelscited by 107
  8. 2020Sequential Latent Knowledge Selection for Knowledge-Grounded Dialoguecited by 82
  9. 2020Beyond Domain APIs: Task-oriented Conversational Modeling with Unstructured Knowledge Accesscited by 58
  10. 2018Towards Exploiting Background Knowledge for Building Conversation Systemscited by 57

Built on by

Cited by year

Citing papers per year: 2021: 9; 2022: 8; 2023: 6; 2024: 9; 2025: 1.

33 papers.

Sort:

By area:

  1. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  2. 2024CBR-RAG: Case-Based Reasoning for Retrieval Augmented Generation in LLMs for Legal Question Answeringcited by 70
  3. 2024LitLLM: A Toolkit for Scientific Literature Reviewcited by 61
  4. 2024RAGChecker: A Fine-grained Framework for Diagnosing Retrieval-Augmented Generationcited by 59
  5. 2021Fast Nearest Neighbor Machine Translationcited by 37
  6. 2024FairRAG: Fair Human Generation via Fair Retrieval Augmentationcited by 34
  7. 2021Reason first, then respond: Modular Generation for Knowledge-infused Dialoguecited by 32
  8. 2021GNN-LM: Language Modeling based on Global Contexts via GNNcited by 27
  9. 2024RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processingcited by 25
  10. 2022Relational Memory Augmented Language Modelscited by 24
  11. 2021RetGen: A Joint framework for Retrieval and Grounded Text Generation Modelingcited by 22
  12. 2024Accelerating Retrieval-Augmented Generationcited by 14
21 more
  1. 2023Simple and Scalable Nearest Neighbor Machine Translationcited by 12
  2. 2024Retrieval-Enhanced Machine Learning: Synthesis and Opportunitiescited by 11
  3. 2021Faster Nearest Neighbor Machine Translationcited by 8
  4. 2022AARGH! End-to-end Retrieval-Generation for Task-Oriented Dialogcited by 8
  5. 2022kNN-NER: Named Entity Recognition with Nearest Neighbor Searchcited by 6
  6. 2021Adapting Document-Grounded Dialog Systems to Spoken Conversations using Data Augmentation and a Noisy Channel Modelcited by 5
  7. 2022OPERA: Harmonizing Task-Oriented Dialogs and Information Seeking Experiencecited by 5
  8. 2022Knowledge-grounded Dialog State Trackingcited by 5
  9. 2021Cascaded Span Extraction and Response Generation for Document-Grounded Dialogcited by 4
  10. 2023Task-oriented Document-Grounded Dialog Systems by HLTPR@RWTH for DSTC9 and DSTC10cited by 4
  11. 2023Revealing User Familiarity Bias in Task-Oriented Dialogue via Interactive Evaluationcited by 4
  12. 2021End-to-End Task-Oriented Dialog Modeling with Semi-Structured Knowledge Managementcited by 3
  13. 2022Controllable Factuality in Document-Grounded Dialog Systems Using a Noisy Channel Modelcited by 3
  14. 2022GNN-SL: Sequence Labeling Based on Nearest Examples via GNNcited by 3
  15. 2024How Does Knowledge Selection Help Retrieval Augmented Generation?cited by 3
  16. 2022Query Enhanced Knowledge-Intensive Conversation via Unsupervised Joint Modelingcited by 2
  17. 2025Roles of MLLMs in Visually Rich Document Retrieval for RAG: A Surveycited by 2
  18. 2023Nearest Neighbor Machine Translation is Meta-Optimizer on Output Projection Layercited by 1
  19. 2023Retrieval-Augmented Neural Response Generation Using Logical Reasoning and Relevance Scoringcited by 1
  20. 2023A Graph-to-Text Approach to Knowledge-Grounded Response Generation in Human-Robot Interactioncited by 0
  21. 2024SAKR: Enhancing Retrieval-Augmented Generation via Streaming Algorithm and K-Means Clusteringcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022KGI: An Integrated Framework for Knowledge Intensive Language Tasks
  2. 2024UniMS-RAG: A Unified Multi-source Retrieval-Augmented Generation for Personalized Dialogue Systems
  3. 2024RAGLAB: A Modular and Research-Oriented Unified Framework for Retrieval-Augmented Generation
  4. 2023Knowledge Graph-Augmented Language Models for Knowledge-Grounded Dialogue Generation
  5. 2025A Survey on Retrieval And Structuring Augmented Generation with Large Language Models
  6. 2025Reasoning in Action: MCTS-Driven Knowledge Retrieval for Large Language Models
  7. 2023Knowledge-Retrieval Task-Oriented Dialog Systems with Semi-Supervision
  8. 2024From RAGs to rich parameters: Probing how language models utilize external knowledge over parametric information for factual queries