# Contextual Knowledge Learning For Dialogue Generation

arXiv: https://arxiv.org/abs/2305.18200 · HTML: https://mlsanity.com/paper/2305.18200/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 46 references that are in the map.
- First posted on arXiv 29 May 2023.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [Retrieval Augmentation Reduces Hallucination in Conversation](https://mlsanity.com/paper/2104.07567/) (2021)
- [Wizard of Wikipedia: Knowledge-Powered Conversational agents](https://mlsanity.com/paper/1811.01241/) (2018)
- [How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation](https://mlsanity.com/paper/1603.08023/) (2016)
- [A Knowledge-Grounded Neural Conversation Model](https://mlsanity.com/paper/1702.01932/) (2017)
- [Knowledge-Grounded Dialogue Generation with Pre-trained Language Models](https://mlsanity.com/paper/2010.08824/) (2020)
- [A Dataset for Document Grounded Conversations](https://mlsanity.com/paper/1809.07358/) (2018)
- [Re2G: Retrieve, Rerank, Generate](https://mlsanity.com/paper/2207.06300/) (2022)
- [Sequential Latent Knowledge Selection for Knowledge-Grounded Dialogue](https://mlsanity.com/paper/2002.07510/) (2020)
- [Learning to Select Knowledge for Response Generation in Dialog Systems](https://mlsanity.com/paper/1902.04911/) (2019)
- [Low-Resource Knowledge-Grounded Dialogue Generation](https://mlsanity.com/paper/2002.10348/) (2020)
- [Do Neural Dialog Systems Use the Conversation History Effectively? An Empirical Study](https://mlsanity.com/paper/1906.01603/) (2019)
- [Hindsight: Posterior-guided training of retrievers for improved open-ended generation](https://mlsanity.com/paper/2110.07752/) (2021)
- [Incremental Transformer with Deliberation Decoder for Document Grounded Conversations](https://mlsanity.com/paper/1907.08854/) (2019)
- [Zero-Resource Knowledge-Grounded Dialogue Generation](https://mlsanity.com/paper/2008.12918/) (2020)
- [Focused Attention Improves Document-Grounded Generation](https://mlsanity.com/paper/2104.12714/) (2021)
- [DIALKI: Knowledge Identification in Conversational Systems through Dialogue-Document Contextualization](https://mlsanity.com/paper/2109.04673/) (2021)
- [A Three-Stage Learning Framework for Low-Resource Knowledge-Grounded Dialogue Generation](https://mlsanity.com/paper/2109.04096/) (2021)
- [Jointly Improving Language Understanding and Generation with Quality-Weighted Weak Supervision of Automatic Labeling](https://mlsanity.com/paper/2102.03551/) (2021)
- [Weak Supervision Enhanced Generative Network for Question Generation](https://mlsanity.com/paper/1907.00607/) (2019)

## Built on by (most cited first, 1 of 1)

- [Unleashing Potential of Evidence in Knowledge-Intensive Dialogue Generation](https://mlsanity.com/paper/2309.08380/) (2023)

## Abstract (arXiv)

Incorporating conversational context and knowledge into dialogue generation models has been essential for improving the quality of the generated responses. The context, comprising utterances from previous dialogue exchanges, is used as a source of content for response generation and as a means of selecting external knowledge. However, to avoid introducing irrelevant content, it is key to enable fine-grained scoring of context and knowledge. In this paper, we present a novel approach to context and knowledge weighting as an integral part of model training. We guide the model training through a Contextual Knowledge Learning (CKL) process which involves Latent Vectors for context and knowledge, respectively. CKL Latent Vectors capture the relationship between context, knowledge, and responses through weak supervision and enable differential weighting of context utterances and knowledge sentences during the training process. Experiments with two standard datasets and human evaluation demonstrate that CKL leads to a significant improvement compared with the performance of six strong baseline models and shows robustness with regard to reduced sizes of training sets.