# Incremental Transformer with Deliberation Decoder for Document Grounded Conversations

arXiv: https://arxiv.org/abs/1907.08854 · HTML: https://mlsanity.com/paper/1907.08854/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jul 2019.

## Built on by (most cited first, 30 of 35)

- [A Survey on Retrieval-Augmented Text Generation](https://mlsanity.com/paper/2202.01110/) (2022)
- [Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey](https://mlsanity.com/paper/2105.04387/) (2021)
- [Grounding 'Grounding' in NLP](https://mlsanity.com/paper/2106.02192/) (2021)
- [Conversations Are Not Flat: Modeling the Dynamic Information Flow across Dialogue Utterances](https://mlsanity.com/paper/2106.02227/) (2021)
- [Rome was built in 1776: A Case Study on Factual Correctness in Knowledge-Grounded Response Generation](https://mlsanity.com/paper/2110.05456/) (2021)
- [DIALKI: Knowledge Identification in Conversational Systems through Dialogue-Document Contextualization](https://mlsanity.com/paper/2109.04673/) (2021)
- [A Three-Stage Learning Framework for Low-Resource Knowledge-Grounded Dialogue Generation](https://mlsanity.com/paper/2109.04096/) (2021)
- [Learning Retrieval Augmentation for Personalized Dialogue Generation](https://mlsanity.com/paper/2406.18847/) (2024)
- [Generative Knowledge Selection for Knowledge-Grounded Dialogues](https://mlsanity.com/paper/2304.04836/) (2023)
- [Knowledge Enhanced Fine-Tuning for Better Handling Unseen Entities in Dialogue Generation](https://mlsanity.com/paper/2109.05487/) (2021)
- [Report from the NSF Future Directions Workshop on Automatic Evaluation of Dialog: Research Directions and Challenges](https://mlsanity.com/paper/2203.10012/) (2022)
- [Multimodal Incremental Transformer with Visual Grounding for Visual Dialogue Generation](https://mlsanity.com/paper/2109.08478/) (2021)
- [Multi-Document Scientific Summarization from a Knowledge Graph-Centric View](https://mlsanity.com/paper/2209.04319/) (2022)
- [Don't Forget Your ABC's: Evaluating the State-of-the-Art in Chat-Oriented Dialogue Systems](https://mlsanity.com/paper/2212.09180/) (2022)
- [Commonsense and Named Entity Aware Knowledge Grounded Dialogue Generation](https://mlsanity.com/paper/2205.13928/) (2022)
- [Opening up Minds with Argumentative Dialogues](https://mlsanity.com/paper/2301.06400/) (2023)
- [Task2Dial: A Novel Task and Dataset for Commonsense enhanced Task-based Dialogue Grounded in Documents](https://mlsanity.com/paper/2204.01061/) (2022)
- [Towards Standard Criteria for human evaluation of Chatbots: A Survey](https://mlsanity.com/paper/2105.11197/) (2021)
- [EZInterviewer: To Improve Job Interview Performance with Mock Interview Generator](https://mlsanity.com/paper/2301.00972/) (2023)
- [Coarse-to-Fine Knowledge Selection for Document Grounded Dialogs](https://mlsanity.com/paper/2302.11849/) (2023)
- [Collaborative Synthesis of Patient Records through Multi-Visit Health State Inference](https://mlsanity.com/paper/2312.14646/) (2023)
- [TAPIR: Learning Adaptive Revision for Incremental Natural Language Understanding with a Two-Pass Model](https://mlsanity.com/paper/2305.10845/) (2023)
- [Sequence-Level Certainty Reduces Hallucination In Knowledge-Grounded Dialogue Generation](https://mlsanity.com/paper/2310.18794/) (2023)
- [Through the Prism of Culture: Evaluating LLMs' Understanding of Indian Subcultures and Traditions](https://mlsanity.com/paper/2501.16748/) (2025)
- [Incremental Deep Neural Network Learning using Classification Confidence Thresholding](https://mlsanity.com/paper/2106.11437/) (2021)
- [How About Kind of Generating Hedges using End-to-End Neural Models?](https://mlsanity.com/paper/2306.14696/) (2023)
- [Building Knowledge-Grounded Dialogue Systems with Graph-Based Semantic Modeling](https://mlsanity.com/paper/2204.12681/) (2022)
- [DialoGen: Generalized Long-Range Context Representation for Dialogue Systems](https://mlsanity.com/paper/2210.06282/) (2022)
- [Converse, Focus and Guess -- Towards Multi-Document Driven Dialogue](https://mlsanity.com/paper/2102.02435/) (2021)
- [Lexical Knowledge Internalization for Neural Dialog Generation](https://mlsanity.com/paper/2205.01941/) (2022)

## Abstract (arXiv)

Document Grounded Conversations is a task to generate dialogue responses when chatting about the content of a given document. Obviously, document knowledge plays a critical role in Document Grounded Conversations, while existing dialogue models do not exploit this kind of knowledge effectively enough. In this paper, we propose a novel Transformer-based architecture for multi-turn document grounded conversations. In particular, we devise an Incremental Transformer to encode multi-turn utterances along with knowledge in related documents. Motivated by the human cognitive process, we design a two-pass decoder (Deliberation Decoder) to improve context coherence and knowledge correctness. Our empirical study on a real-world Document Grounded Dataset proves that responses generated by our model significantly outperform competitive baselines on both context coherence and knowledge relevance.