# Grounding Dialogue Systems via Knowledge Graph Aware Decoding with Pre-trained Transformers

arXiv: https://arxiv.org/abs/2103.16289 · HTML: https://mlsanity.com/paper/2103.16289/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 41 references that are in the map.
- First posted on arXiv 30 Mar 2021.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Geometric deep learning: going beyond Euclidean data](https://mlsanity.com/paper/1611.08097/) (2016)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [Learning Deep Transformer Models for Machine Translation](https://mlsanity.com/paper/1906.01787/) (2019)
- [A Network-based End-to-End Trainable Task-oriented Dialogue System](https://mlsanity.com/paper/1604.04562/) (2016)
- [Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders](https://mlsanity.com/paper/1703.10960/) (2017)
- [Key-Value Retrieval Networks for Task-Oriented Dialogue](https://mlsanity.com/paper/1705.05414/) (2017)
- [Large-scale Simple Question Answering with Memory Networks](https://mlsanity.com/paper/1506.02075/) (2015)
- [Learning End-to-End Goal-Oriented Dialog](https://mlsanity.com/paper/1605.07683/) (2016)
- [Mem2Seq: Effectively Incorporating Knowledge Bases into End-to-End Task-Oriented Dialog Systems](https://mlsanity.com/paper/1804.08217/) (2018)
- [Relevance of Unsupervised Metrics in Task-Oriented Dialogue for Evaluating Natural Language Generation](https://mlsanity.com/paper/1706.09799/) (2017)
- [Global-to-local Memory Pointer Networks for Task-Oriented Dialogue](https://mlsanity.com/paper/1901.04713/) (2019)
- [Improved Neural Relation Detection for Knowledge Base Question Answering](https://mlsanity.com/paper/1704.06194/) (2017)
- [Towards End-to-End Reinforcement Learning of Dialogue Agents for Information Access](https://mlsanity.com/paper/1609.00777/) (2016)
- [SimpleQuestions Nearly Solved: A New Upperbound and Baseline Approach](https://mlsanity.com/paper/1804.08798/) (2018)
- [Introduction to Neural Network based Approaches for Question Answering over Knowledge Graphs](https://mlsanity.com/paper/1907.09361/) (2019)
- [Hierarchical Transformer for Task Oriented Dialog Systems](https://mlsanity.com/paper/2011.08067/) (2020)
- [Strong Baselines for Simple Question Answering over Knowledge Graphs with and without Neural Networks](https://mlsanity.com/paper/1712.01969/) (2017)
- [Improving Variational Autoencoder for Text Modelling with Timestep-Wise Regularisation](https://mlsanity.com/paper/2011.01136/) (2020)
- [Learning Representation Mapping for Relation Detection in Knowledge Base Question Answering](https://mlsanity.com/paper/1907.07328/) (2019)
- [Multi-turn Dialogue Response Generation in an Adversarial Learning Framework](https://mlsanity.com/paper/1805.11752/) (2018)
- [Using a KG-Copy Network for Non-Goal Oriented Dialogues](https://mlsanity.com/paper/1910.07834/) (2019)
- [Interpretable Graph Convolutional Neural Networks for Inference on Noisy Knowledge Graphs](https://mlsanity.com/paper/1812.00279/) (2018)
- [Incorporating Joint Embeddings into Goal-Oriented Dialogues with Multi-Task Learning](https://mlsanity.com/paper/2001.10468/) (2020)

## Built on by (most cited first, 8 of 8)

- [RHO (ρ): Reducing Hallucination in Open-domain Dialogues with Knowledge Grounding](https://mlsanity.com/paper/2212.01588/) (2022)
- [DialoKG: Knowledge-Structure Aware Task-Oriented Dialogue Generation](https://mlsanity.com/paper/2204.09149/) (2022)
- [CarExpert: Leveraging Large Language Models for In-Car Conversational Question Answering](https://mlsanity.com/paper/2310.09536/) (2023)
- [RoMe: A Robust Metric for Evaluating Natural Language Generation](https://mlsanity.com/paper/2203.09183/) (2022)
- [Efficient and Transferable Agentic Knowledge Graph RAG via Reinforcement Learning](https://mlsanity.com/paper/2509.26383/) (2025)
- [From Data to Dialogue: Leveraging the Structure of Knowledge Graphs for Conversational Exploratory Search](https://mlsanity.com/paper/2310.05150/) (2023)
- [Mind the Knowledge Gap: A Survey of Knowledge-enhanced Dialogue Systems](https://mlsanity.com/paper/2212.09252/) (2022)
- [Voice-Based Conversational Agents and Knowledge Graphs for Improving News Search in Assisted Living](https://mlsanity.com/paper/2303.14286/) (2023)

## Abstract (arXiv)

Generating knowledge grounded responses in both goal and non-goal oriented dialogue systems is an important research challenge. Knowledge Graphs (KG) can be viewed as an abstraction of the real world, which can potentially facilitate a dialogue system to produce knowledge grounded responses. However, integrating KGs into the dialogue generation process in an end-to-end manner is a non-trivial task. This paper proposes a novel architecture for integrating KGs into the response generation process by training a BERT model that learns to answer using the elements of the KG (entities and relations) in a multi-task, end-to-end setting. The k-hop subgraph of the KG is incorporated into the model during training and inference using Graph Laplacian. Empirical evaluation suggests that the model achieves better knowledge groundedness (measured via Entity F1 score) compared to other state-of-the-art models for both goal and non-goal oriented dialogues.