# Knowledge-Grounded Dialogue Generation with a Unified Knowledge Representation

arXiv: https://arxiv.org/abs/2112.07924 · HTML: https://mlsanity.com/paper/2112.07924/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 55 references that are in the map.
- First posted on arXiv 15 Dec 2021.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [How Much Knowledge Can You Pack Into the Parameters of a Language Model?](https://mlsanity.com/paper/2002.08910/) (2020)
- [Retrieval Augmentation Reduces Hallucination in Conversation](https://mlsanity.com/paper/2104.07567/) (2021)
- [Recipes for building an open-domain chatbot](https://mlsanity.com/paper/2004.13637/) (2020)
- [Wizard of Wikipedia: Knowledge-Powered Conversational agents](https://mlsanity.com/paper/1811.01241/) (2018)
- [Towards a Human-like Open-Domain Chatbot](https://mlsanity.com/paper/2001.09977/) (2020)
- [A Simple Language Model for Task-Oriented Dialogue](https://mlsanity.com/paper/2005.00796/) (2020)
- [Towards Deep Conversational Recommendations](https://mlsanity.com/paper/1812.07617/) (2018)
- [The Design and Implementation of XiaoIce, an Empathetic Social Chatbot](https://mlsanity.com/paper/1812.08989/) (2018)
- [A Knowledge-Grounded Neural Conversation Model](https://mlsanity.com/paper/1702.01932/) (2017)
- [Improving Conversational Recommender Systems via Knowledge Graph based Semantic Fusion](https://mlsanity.com/paper/2007.04032/) (2020)
- [Towards Knowledge-Based Recommender Dialog System](https://mlsanity.com/paper/1908.05391/) (2019)
- [Knowledge-Grounded Dialogue Generation with Pre-trained Language Models](https://mlsanity.com/paper/2010.08824/) (2020)
- [A Dataset for Document Grounded Conversations](https://mlsanity.com/paper/1809.07358/) (2018)
- [INSPIRED: Toward Sociable Recommendation Dialog Systems](https://mlsanity.com/paper/2009.14306/) (2020)
- [SOLOIST: Building Task Bots at Scale with Transfer Learning and Machine Teaching](https://mlsanity.com/paper/2005.05298/) (2020)
- [Low-Resource Knowledge-Grounded Dialogue Generation](https://mlsanity.com/paper/2002.10348/) (2020)
- [Knowledge-Aware Language Model Pretraining](https://mlsanity.com/paper/2007.00655/) (2020)
- [DuRecDial 2.0: A Bilingual Parallel Corpus for Conversational Recommendation](https://mlsanity.com/paper/2109.08877/) (2021)

## Built on by (most cited first, 30 of 32)

- [Retrieving Multimodal Information for Augmented Generation: A Survey](https://mlsanity.com/paper/2303.10868/) (2023)
- [Cognitive Mirage: A Review of Hallucinations in Large Language Models](https://mlsanity.com/paper/2309.06794/) (2023)
- [A Survey of Knowledge Enhanced Pre-trained Language Models](https://mlsanity.com/paper/2211.05994/) (2022)
- [Knowledge Graph-Augmented Language Models for Knowledge-Grounded Dialogue Generation](https://mlsanity.com/paper/2305.18846/) (2023)
- [RHO (ρ): Reducing Hallucination in Open-domain Dialogues with Knowledge Grounding](https://mlsanity.com/paper/2212.01588/) (2022)
- [Graph Reasoning for Question Answering with Triplet Retrieval](https://mlsanity.com/paper/2305.18742/) (2023)
- [EVA2.0: Investigating Open-Domain Chinese Dialogue Systems with Large-Scale Pre-Training](https://mlsanity.com/paper/2203.09313/) (2022)
- [Adaptive Retrieval-Augmented Generation for Conversational Systems](https://mlsanity.com/paper/2407.21712/) (2024)
- [Can Language Models Act as Knowledge Bases at Scale?](https://mlsanity.com/paper/2402.14273/) (2024)
- [On Zero-Shot Counterspeech Generation by LLMs](https://mlsanity.com/paper/2403.14938/) (2024)
- [Unleashing Potential of Evidence in Knowledge-Intensive Dialogue Generation](https://mlsanity.com/paper/2309.08380/) (2023)
- [A review of faithfulness metrics for hallucination assessment in Large Language Models](https://mlsanity.com/paper/2501.00269/) (2024)
- [Script-Strategy Aligned Generation: Aligning LLMs with Expert-Crafted Dialogue Scripts and Therapeutic Strategies for Psychotherapy](https://mlsanity.com/paper/2411.06723/) (2024)
- [PK-ICR: Persona-Knowledge Interactive Context Retrieval for Grounded Dialogue](https://mlsanity.com/paper/2302.06674/) (2023)
- [KnowPrefix-Tuning: A Two-Stage Prefix-Tuning Framework for Knowledge-Grounded Dialogue Generation](https://mlsanity.com/paper/2306.15430/) (2023)
- [Sequence-Level Certainty Reduces Hallucination In Knowledge-Grounded Dialogue Generation](https://mlsanity.com/paper/2310.18794/) (2023)
- [CrowdCounter: A benchmark type-specific multi-target counterspeech dataset](https://mlsanity.com/paper/2410.01400/) (2024)
- [INSPIRED2: An Improved Dataset for Sociable Conversational Recommendation](https://mlsanity.com/paper/2208.04104/) (2022)
- [MOCHA: A Multi-Task Training Approach for Coherent Text Generation from Cognitive Perspective](https://mlsanity.com/paper/2210.14650/) (2022)
- [DialGuide: Aligning Dialogue Model Behavior with Developer Guidelines](https://mlsanity.com/paper/2212.10557/) (2022)
- [The Open-domain Paradox for Chatbots: Common Ground as the Basis for Human-like Dialogue](https://mlsanity.com/paper/2303.11708/) (2023)
- [Causal Document-Grounded Dialogue Pre-training](https://mlsanity.com/paper/2305.10927/) (2023)
- [TROVE: A Challenge for Fine-Grained Text Provenance via Source Sentence Tracing and Relationship Classification](https://mlsanity.com/paper/2503.15289/) (2025)
- [A Conversation is Worth A Thousand Recommendations: A Survey of Holistic Conversational Recommender Systems](https://mlsanity.com/paper/2309.07682/) (2023)
- [Building Knowledge-Grounded Dialogue Systems with Graph-Based Semantic Modeling](https://mlsanity.com/paper/2204.12681/) (2022)
- [Improving the Robustness of Knowledge-Grounded Dialogue via Contrastive Learning](https://mlsanity.com/paper/2401.04361/) (2024)
- [Roles of MLLMs in Visually Rich Document Retrieval for RAG: A Survey](https://mlsanity.com/paper/2601.03262/) (2025)
- [Mind the Knowledge Gap: A Survey of Knowledge-enhanced Dialogue Systems](https://mlsanity.com/paper/2212.09252/) (2022)
- [PICK: Polished & Informed Candidate Scoring for Knowledge-Grounded Dialogue Systems](https://mlsanity.com/paper/2309.10413/) (2023)
- [Graph vs. Sequence: An Empirical Study on Knowledge Forms for Knowledge-Grounded Dialogue](https://mlsanity.com/paper/2312.07868/) (2023)

## Abstract (arXiv)

Knowledge-grounded dialogue systems are challenging to build due to the lack of training data and heterogeneous knowledge sources. Existing systems perform poorly on unseen topics due to limited topics covered in the training data. In addition, heterogeneous knowledge sources make it challenging for systems to generalize to other tasks because knowledge sources in different knowledge representations require different knowledge encoders. To address these challenges, we present PLUG, a language model that homogenizes different knowledge sources to a unified knowledge representation for knowledge-grounded dialogue generation tasks. PLUG is pre-trained on a dialogue generation task conditioned on a unified essential knowledge representation. It can generalize to different downstream knowledge-grounded dialogue generation tasks with a few training examples. The empirical evaluation on two benchmarks shows that our model generalizes well across different knowledge-grounded tasks. It can achieve comparable performance with state-of-the-art methods under a fully-supervised setting and significantly outperforms other methods in zero-shot and few-shot settings.