# Contextualize Knowledge Bases with Transformer for End-to-end Task-Oriented Dialogue Systems

arXiv: https://arxiv.org/abs/2010.05740 · HTML: https://mlsanity.com/paper/2010.05740/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Oct 2020.

## Built on by (most cited first, 8 of 8)

- [UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models](https://mlsanity.com/paper/2201.05966/) (2022)
- [A Survey on Text-to-SQL Parsing: Concepts, Methods, and Future Directions](https://mlsanity.com/paper/2208.13629/) (2022)
- [End-to-end Task-oriented Dialogue: A Survey of Tasks, Methods, and Future Directions](https://mlsanity.com/paper/2311.09008/) (2023)
- [Towards Large-Scale Interpretable Knowledge Graph Reasoning for Dialogue Systems](https://mlsanity.com/paper/2203.10610/) (2022)
- [Q-TOD: A Query-driven Task-oriented Dialogue System](https://mlsanity.com/paper/2210.07564/) (2022)
- [Leveraging class abstraction for commonsense reinforcement learning via residual policy gradient methods](https://mlsanity.com/paper/2201.12126/) (2022)
- [Combining Transformers with Natural Language Explanations](https://mlsanity.com/paper/2110.00125/) (2021)
- [Mind the Knowledge Gap: A Survey of Knowledge-enhanced Dialogue Systems](https://mlsanity.com/paper/2212.09252/) (2022)

## Abstract (arXiv)

Incorporating knowledge bases (KB) into end-to-end task-oriented dialogue systems is challenging, since it requires to properly represent the entity of KB, which is associated with its KB context and dialogue context. The existing works represent the entity with only perceiving a part of its KB context, which can lead to the less effective representation due to the information loss, and adversely favor KB reasoning and response generation. To tackle this issue, we explore to fully contextualize the entity representation by dynamically perceiving all the relevant entities} and dialogue history. To achieve this, we propose a COntext-aware Memory Enhanced Transformer framework (COMET), which treats the KB as a sequence and leverages a novel Memory Mask to enforce the entity to only focus on its relevant entities and dialogue history, while avoiding the distraction from the irrelevant entities. Through extensive experiments, we show that our COMET framework can achieve superior performance over the state of the arts.