# A Three-Stage Learning Framework for Low-Resource Knowledge-Grounded Dialogue Generation

arXiv: https://arxiv.org/abs/2109.04096 · HTML: https://mlsanity.com/paper/2109.04096/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 39 references that are in the map.
- First posted on arXiv 9 Sep 2021.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering](https://mlsanity.com/paper/2007.01282/) (2020)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [Unified Language Model Pre-training for Natural Language Understanding and Generation](https://mlsanity.com/paper/1905.03197/) (2019)
- [Wizard of Wikipedia: Knowledge-Powered Conversational agents](https://mlsanity.com/paper/1811.01241/) (2018)
- [A Neural Conversational Model](https://mlsanity.com/paper/1506.05869/) (2015)
- [ParlAI: A Dialog Research Software Platform](https://mlsanity.com/paper/1705.06476/) (2017)
- [A Knowledge-Grounded Neural Conversation Model](https://mlsanity.com/paper/1702.01932/) (2017)
- [Knowledge-Grounded Dialogue Generation with Pre-trained Language Models](https://mlsanity.com/paper/2010.08824/) (2020)
- [A Dataset for Document Grounded Conversations](https://mlsanity.com/paper/1809.07358/) (2018)
- [Proactive Human-Machine Conversation with Explicit Conversation Goals](https://mlsanity.com/paper/1906.05572/) (2019)
- [Sequential Latent Knowledge Selection for Knowledge-Grounded Dialogue](https://mlsanity.com/paper/2002.07510/) (2020)
- [Learning to Select Knowledge for Response Generation in Dialog Systems](https://mlsanity.com/paper/1902.04911/) (2019)
- [Low-Resource Knowledge-Grounded Dialogue Generation](https://mlsanity.com/paper/2002.10348/) (2020)
- [Incremental Transformer with Deliberation Decoder for Document Grounded Conversations](https://mlsanity.com/paper/1907.08854/) (2019)
- [Zero-Resource Knowledge-Grounded Dialogue Generation](https://mlsanity.com/paper/2008.12918/) (2020)
- [DyKgChat: Benchmarking Dialogue Generation Grounding on Dynamic Knowledge Graphs](https://mlsanity.com/paper/1910.00610/) (2019)
- [Augmenting Neural Response Generation with Context-Aware Topical Attention](https://mlsanity.com/paper/1811.01063/) (2018)
- [Guided Transformer: Leveraging Multiple External Sources for Representation Learning in Conversational Search](https://mlsanity.com/paper/2006.07548/) (2020)
- [DeepCopy: Grounded Response Generation with Hierarchical Pointer Networks](https://mlsanity.com/paper/1908.10731/) (2019)
- [Disentangling Language and Knowledge in Task-Oriented Dialogs](https://mlsanity.com/paper/1805.01216/) (2018)
- [Meta-Transfer Learning for Low-Resource Abstractive Summarization](https://mlsanity.com/paper/2102.09397/) (2021)

## Built on by (most cited first, 19 of 19)

- [Pretrained Language Models for Text Generation: A Survey](https://mlsanity.com/paper/2201.05273/) (2022)
- [Knowledge-Grounded Dialogue Generation with a Unified Knowledge Representation](https://mlsanity.com/paper/2112.07924/) (2021)
- [RECAP: Retrieval-Enhanced Context-Aware Prefix Encoder for Personalized Dialogue Response Generation](https://mlsanity.com/paper/2306.07206/) (2023)
- [Exploring Prompt-based Few-shot Learning for Grounded Dialog Generation](https://mlsanity.com/paper/2109.06513/) (2021)
- [KILM: Knowledge Injection into Encoder-Decoder Language Models](https://mlsanity.com/paper/2302.09170/) (2023)
- [Multimedia Generative Script Learning for Task Planning](https://mlsanity.com/paper/2208.12306/) (2022)
- [PK-ICR: Persona-Knowledge Interactive Context Retrieval for Grounded Dialogue](https://mlsanity.com/paper/2302.06674/) (2023)
- [Diverse and Faithful Knowledge-Grounded Dialogue Generation via Sequential Posterior Inference](https://mlsanity.com/paper/2306.01153/) (2023)
- [KGQuiz: Evaluating the Generalization of Encoded Knowledge in Large Language Models](https://mlsanity.com/paper/2310.09725/) (2023)
- [Building Knowledge-Grounded Dialogue Systems with Graph-Based Semantic Modeling](https://mlsanity.com/paper/2204.12681/) (2022)
- [Breaking the Trade-Off Between Faithfulness and Expressiveness for Large Language Models](https://mlsanity.com/paper/2508.18651/) (2025)
- [Mind the Knowledge Gap: A Survey of Knowledge-enhanced Dialogue Systems](https://mlsanity.com/paper/2212.09252/) (2022)
- [Position Matters! Empirical Study of Order Effect in Knowledge-grounded Dialogue](https://mlsanity.com/paper/2302.05888/) (2023)
- [Contextual Knowledge Learning For Dialogue Generation](https://mlsanity.com/paper/2305.18200/) (2023)
- [PICK: Polished & Informed Candidate Scoring for Knowledge-Grounded Dialogue Systems](https://mlsanity.com/paper/2309.10413/) (2023)
- [Graph vs. Sequence: An Empirical Study on Knowledge Forms for Knowledge-Grounded Dialogue](https://mlsanity.com/paper/2312.07868/) (2023)
- [A Factuality and Diversity Reconciled Decoding Method for Knowledge-Grounded Dialogue Generation](https://mlsanity.com/paper/2407.05718/) (2024)
- [ChatZero:Zero-shot Cross-Lingual Dialogue Generation via Pseudo-Target Language](https://mlsanity.com/paper/2408.08724/) (2024)
- [Efficient Tuning of Large Language Models for Knowledge-Grounded Dialogue Generation](https://mlsanity.com/paper/2504.07754/) (2025)

## Abstract (arXiv)

Neural conversation models have shown great potentials towards generating fluent and informative responses by introducing external background knowledge. Nevertheless, it is laborious to construct such knowledge-grounded dialogues, and existing models usually perform poorly when transfer to new domains with limited training samples. Therefore, building a knowledge-grounded dialogue system under the low-resource setting is a still crucial issue. In this paper, we propose a novel three-stage learning framework based on weakly supervised learning which benefits from large scale ungrounded dialogues and unstructured knowledge base. To better cooperate with this framework, we devise a variant of Transformer with decoupled decoder which facilitates the disentangled learning of response generation and knowledge incorporation. Evaluation results on two benchmarks indicate that our approach can outperform other state-of-the-art methods with less training data, and even in zero-resource scenario, our approach still performs well.