# DialoKG: Knowledge-Structure Aware Task-Oriented Dialogue Generation

arXiv: https://arxiv.org/abs/2204.09149 · HTML: https://mlsanity.com/paper/2204.09149/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 49 references that are in the map.
- First posted on arXiv 19 Apr 2022.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [Pointer Networks](https://mlsanity.com/paper/1506.03134/) (2015)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance](https://mlsanity.com/paper/1909.02622/) (2019)
- [QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering](https://mlsanity.com/paper/2104.06378/) (2021)
- [How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation](https://mlsanity.com/paper/1603.08023/) (2016)
- [A Simple Language Model for Task-Oriented Dialogue](https://mlsanity.com/paper/2005.00796/) (2020)
- [A Network-based End-to-End Trainable Task-oriented Dialogue System](https://mlsanity.com/paper/1604.04562/) (2016)
- [Why We Need New Evaluation Metrics for NLG](https://mlsanity.com/paper/1707.06875/) (2017)
- [Incorporating Copying Mechanism in Sequence-to-Sequence Learning](https://mlsanity.com/paper/1603.06393/) (2016)
- [PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variable](https://mlsanity.com/paper/1910.07931/) (2019)
- [Key-Value Retrieval Networks for Task-Oriented Dialogue](https://mlsanity.com/paper/1705.05414/) (2017)
- [A Knowledge-Grounded Neural Conversation Model](https://mlsanity.com/paper/1702.01932/) (2017)
- [Sequential Latent Knowledge Selection for Knowledge-Grounded Dialogue](https://mlsanity.com/paper/2002.07510/) (2020)
- [Mem2Seq: Effectively Incorporating Knowledge Bases into End-to-End Task-Oriented Dialog Systems](https://mlsanity.com/paper/1804.08217/) (2018)
- [Table-to-text Generation by Structure-aware Seq2seq Learning](https://mlsanity.com/paper/1711.09724/) (2017)
- [Global-to-local Memory Pointer Networks for Task-Oriented Dialogue](https://mlsanity.com/paper/1901.04713/) (2019)
- [Dynamic Fusion Network for Multi-Domain End-to-end Task-Oriented Dialog](https://mlsanity.com/paper/2004.11019/) (2020)
- [Learning Knowledge Bases with Parameters for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2009.13656/) (2020)
- [Entity-Consistent End-to-end Task-Oriented Dialogue System with KB Retriever](https://mlsanity.com/paper/1909.06762/) (2019)
- [Sequence-to-Sequence Learning for Task-oriented Dialogue with Dialogue State Representation](https://mlsanity.com/paper/1806.04441/) (2018)
- [Multi-level Memory for Task Oriented Dialogs](https://mlsanity.com/paper/1810.10647/) (2018)
- [Constraint based Knowledge Base Distillation in End-to-End Task Oriented Dialogs](https://mlsanity.com/paper/2109.07396/) (2021)
- [Grounding Dialogue Systems via Knowledge Graph Aware Decoding with Pre-trained Transformers](https://mlsanity.com/paper/2103.16289/) (2021)
- [Confidence-based Graph Convolutional Networks for Semi-Supervised Learning](https://mlsanity.com/paper/1901.08255/) (2019)

## Built on by (most cited first, 19 of 19)

- [Corrective Retrieval Augmented Generation](https://mlsanity.com/paper/2401.15884/) (2024)
- [Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answering](https://mlsanity.com/paper/2306.04136/) (2023)
- [FactKB: Generalizable Factuality Evaluation using Language Models Enhanced with Factual Knowledge](https://mlsanity.com/paper/2305.08281/) (2023)
- [Knowledge Graph-Augmented Language Models for Knowledge-Grounded Dialogue Generation](https://mlsanity.com/paper/2305.18846/) (2023)
- [RHO (ρ): Reducing Hallucination in Open-domain Dialogues with Knowledge Grounding](https://mlsanity.com/paper/2212.01588/) (2022)
- [GenTKG: Generative Forecasting on Temporal Knowledge Graph with Large Language Models](https://mlsanity.com/paper/2310.07793/) (2023)
- [RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processing](https://mlsanity.com/paper/2404.19543/) (2024)
- [DR-RAG: Applying Dynamic Document Relevance to Retrieval-Augmented Generation for Question-Answering](https://mlsanity.com/paper/2406.07348/) (2024)
- [CarExpert: Leveraging Large Language Models for In-Car Conversational Question Answering](https://mlsanity.com/paper/2310.09536/) (2023)
- [KALM: Knowledge-Aware Integration of Local, Document, and Global Contexts for Long Document Understanding](https://mlsanity.com/paper/2210.04105/) (2022)
- [Multi-Grained Knowledge Retrieval for End-to-End Task-Oriented Dialog](https://mlsanity.com/paper/2305.10149/) (2023)
- [Turning Conversations into Workflows: A Framework to Extract and Evaluate Dialog Workflows for Service AI Agents](https://mlsanity.com/paper/2502.17321/) (2025)
- [The Lifecycle of "Facts": A Survey of Social Bias in Knowledge Graphs](https://mlsanity.com/paper/2210.03353/) (2022)
- [Synergizing In-context Learning with Hints for End-to-end Task-oriented Dialog Systems](https://mlsanity.com/paper/2405.15585/) (2024)
- [Retrieval-Generation Alignment for End-to-End Task-Oriented Dialogue System](https://mlsanity.com/paper/2310.08877/) (2023)
- [Dual-Feedback Knowledge Retrieval for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2310.14528/) (2023)
- [MSG-BART: Multi-granularity Scene Graph-Enhanced Encoder-Decoder Language Model for Video-grounded Dialogue Generation](https://mlsanity.com/paper/2311.12820/) (2023)
- [One Subgraph for All: Efficient Reasoning on Opening Subgraphs for Inductive Knowledge Graph Completion](https://mlsanity.com/paper/2404.15807/) (2024)
- [Improving LLM's Attachment to External Knowledge In Dialogue Generation Tasks Through Entity Anonymization](https://mlsanity.com/paper/2511.11946/) (2025)

## Abstract (arXiv)

Task-oriented dialogue generation is challenging since the underlying knowledge is often dynamic and effectively incorporating knowledge into the learning process is hard. It is particularly challenging to generate both human-like and informative responses in this setting. Recent research primarily focused on various knowledge distillation methods where the underlying relationship between the facts in a knowledge base is not effectively captured. In this paper, we go one step further and demonstrate how the structural information of a knowledge graph can improve the system's inference capabilities. Specifically, we propose DialoKG, a novel task-oriented dialogue system that effectively incorporates knowledge into a language model. Our proposed system views relational knowledge as a knowledge graph and introduces (1) a structure-aware knowledge embedding technique, and (2) a knowledge graph-weighted attention masking strategy to facilitate the system selecting relevant information during the dialogue generation. An empirical evaluation demonstrates the effectiveness of DialoKG over state-of-the-art methods on several standard benchmark datasets.