# Analyzing the Forgetting Problem in the Pretrain-Finetuning of Dialogue Response Models

arXiv: https://arxiv.org/abs/1910.07117 · HTML: https://mlsanity.com/paper/1910.07117/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Oct 2019.

## Built on by (most cited first, 4 of 4)

- [Role Prompting Guided Domain Adaptation with General Capability Preserve for Large Language Models](https://mlsanity.com/paper/2403.02756/) (2024)
- [Train-Attention: Meta-Learning Where to Focus in Continual Knowledge Learning](https://mlsanity.com/paper/2407.16920/) (2024)
- [Multi-VQG: Generating Engaging Questions for Multiple Images](https://mlsanity.com/paper/2211.07441/) (2022)
- [Taming the Beast: Learning to Control Neural Conversational Models](https://mlsanity.com/paper/2108.10561/) (2021)

## Abstract (arXiv)

In this work, we study how the finetuning stage in the pretrain-finetune framework changes the behavior of a pretrained neural language generator. We focus on the transformer encoder-decoder model for the open-domain dialogue response generation task. Our major finding is that after standard finetuning, the model forgets some of the important language generation skills acquired during large-scale pretraining. We demonstrate the forgetting phenomenon through a set of detailed behavior analysis from the perspectives of knowledge transfer, context sensitivity, and function space projection. As a preliminary attempt to alleviate the forgetting problem, we propose an intuitive finetuning strategy named "mix-review". We find that mix-review effectively regularizes the finetuning process, and the forgetting problem is alleviated to some extent. Finally, we discuss interesting behavior of the resulting dialogue model and its implications.