# Mem2Seq: Effectively Incorporating Knowledge Bases into End-to-End Task-Oriented Dialog Systems

arXiv: https://arxiv.org/abs/1804.08217 · HTML: https://mlsanity.com/paper/1804.08217/

## Key facts

- Cited by 70 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Apr 2018.

## Built on by (most cited first, 30 of 70)

- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models](https://mlsanity.com/paper/2201.05966/) (2022)
- [Few-Shot Bot: Prompt-Based Learning for Dialogue Systems](https://mlsanity.com/paper/2110.08118/) (2021)
- [BiToD: A Bilingual Multi-Domain Dataset For Task-Oriented Dialogue Modeling](https://mlsanity.com/paper/2106.02787/) (2021)
- [HiURE: Hierarchical Exemplar Contrastive Learning for Unsupervised Relation Extraction](https://mlsanity.com/paper/2205.02225/) (2022)
- [Ask an Expert: Leveraging Language Models to Improve Strategic Reasoning in Goal-Oriented Dialogue Models](https://mlsanity.com/paper/2305.17878/) (2023)
- [MMChat: Multi-Modal Chat Dataset on Social Media](https://mlsanity.com/paper/2108.07154/) (2021)
- [Continual Learning for Task-oriented Dialogue System with Iterative Network Pruning, Expanding and Masking](https://mlsanity.com/paper/2107.08173/) (2021)
- [DisenKGAT: Knowledge Graph Embedding with Disentangled Graph Attention Network](https://mlsanity.com/paper/2108.09628/) (2021)
- [Learning to Remember Patterns: Pattern Matching Memory Networks for Traffic Forecasting](https://mlsanity.com/paper/2110.10380/) (2021)
- [DialoKG: Knowledge-Structure Aware Task-Oriented Dialogue Generation](https://mlsanity.com/paper/2204.09149/) (2022)
- [Don't be Contradicted with Anything! CI-ToD: Towards Benchmarking Consistency for Task-oriented Dialogue System](https://mlsanity.com/paper/2109.11292/) (2021)
- [A Survey on LLM-based Conversational User Simulation](https://mlsanity.com/paper/2604.24977/) (2026)
- [Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2104.08570/) (2021)
- [End-to-end Task-oriented Dialogue: A Survey of Tasks, Methods, and Future Directions](https://mlsanity.com/paper/2311.09008/) (2023)
- [KECRS: Towards Knowledge-Enriched Conversational Recommendation System](https://mlsanity.com/paper/2105.08261/) (2021)
- [Task-Oriented Dialogue System as Natural Language Generation](https://mlsanity.com/paper/2108.13679/) (2021)
- [Dialogue Planning via Brownian Bridge Stochastic Process for Goal-directed Proactive Dialogue](https://mlsanity.com/paper/2305.05290/) (2023)
- [Knowledge-driven Natural Language Understanding of English Text and its Applications](https://mlsanity.com/paper/2101.11707/) (2021)
- [Prix-LM: Pretraining for Multilingual Knowledge Base Construction](https://mlsanity.com/paper/2110.08443/) (2021)
- [Constraint based Knowledge Base Distillation in End-to-End Task Oriented Dialogs](https://mlsanity.com/paper/2109.07396/) (2021)
- [Multimodal Dialog Systems with Dual Knowledge-enhanced Generative Pretrained Language Model](https://mlsanity.com/paper/2207.07934/) (2022)
- [Towards Large-Scale Interpretable Knowledge Graph Reasoning for Dialogue Systems](https://mlsanity.com/paper/2203.10610/) (2022)
- [A Cooperative Memory Network for Personalized Task-oriented Dialogue Systems with Incomplete User Profiles](https://mlsanity.com/paper/2102.08322/) (2021)
- [Grounding Dialogue Systems via Knowledge Graph Aware Decoding with Pre-trained Transformers](https://mlsanity.com/paper/2103.16289/) (2021)
- [Detecting Speaker Personas from Conversational Texts](https://mlsanity.com/paper/2109.01330/) (2021)
- [Autoregressive Entity Generation for End-to-End Task-Oriented Dialog](https://mlsanity.com/paper/2209.08708/) (2022)
- [TILDE-Q: A Transformation Invariant Loss Function for Time-Series Forecasting](https://mlsanity.com/paper/2210.15050/) (2022)
- [Schema-Guided User Satisfaction Modeling for Task-Oriented Dialogues](https://mlsanity.com/paper/2305.16798/) (2023)
- [Fine-grained Contrastive Learning for Relation Extraction](https://mlsanity.com/paper/2205.12491/) (2022)

## Abstract (arXiv)

End-to-end task-oriented dialog systems usually suffer from the challenge of incorporating knowledge bases. In this paper, we propose a novel yet simple end-to-end differentiable model called memory-to-sequence (Mem2Seq) to address this issue. Mem2Seq is the first neural generative model that combines the multi-hop attention over memories with the idea of pointer network. We empirically show how Mem2Seq controls each generation step, and how its multi-hop attention mechanism helps in learning correlations between memories. In addition, our model is quite general without complicated task-specific designs. As a result, we show that Mem2Seq can be trained faster and attain the state-of-the-art performance on three different task-oriented dialog datasets.