# There Are a Thousand Hamlets in a Thousand People's Eyes: Enhancing Knowledge-grounded Dialogue with Personal Memory

arXiv: https://arxiv.org/abs/2204.02624 · HTML: https://mlsanity.com/paper/2204.02624/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 48 references that are in the map.
- First posted on arXiv 6 Apr 2022.

## Builds on (most cited first, 25 of 25)

- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [Personalizing Dialogue Agents: I have a dog, do you have pets too?](https://mlsanity.com/paper/1801.07243/) (2018)
- [Wizard of Wikipedia: Knowledge-Powered Conversational agents](https://mlsanity.com/paper/1811.01241/) (2018)
- [Visual Dialog](https://mlsanity.com/paper/1611.08669/) (2016)
- [DualGAN: Unsupervised Dual Learning for Image-to-Image Translation](https://mlsanity.com/paper/1704.02510/) (2017)
- [The Pushshift Reddit Dataset](https://mlsanity.com/paper/2001.08435/) (2020)
- [Deep Reinforcement Learning for Dialogue Generation](https://mlsanity.com/paper/1606.01541/) (2016)
- [TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents](https://mlsanity.com/paper/1901.08149/) (2019)
- [Dual Learning for Machine Translation](https://mlsanity.com/paper/1611.00179/) (2016)
- [A Knowledge-Grounded Neural Conversation Model](https://mlsanity.com/paper/1702.01932/) (2017)
- [Knowledge-Grounded Dialogue Generation with Pre-trained Language Models](https://mlsanity.com/paper/2010.08824/) (2020)
- [A Dataset for Document Grounded Conversations](https://mlsanity.com/paper/1809.07358/) (2018)
- [Proactive Human-Machine Conversation with Explicit Conversation Goals](https://mlsanity.com/paper/1906.05572/) (2019)
- [You Impress Me: Dialogue Generation via Mutual Persona Perception](https://mlsanity.com/paper/2004.05388/) (2020)
- [Sequential Latent Knowledge Selection for Knowledge-Grounded Dialogue](https://mlsanity.com/paper/2002.07510/) (2020)
- [BoB: BERT Over BERT for Training Persona-based Dialogue Models from Limited Personalized Data](https://mlsanity.com/paper/2106.06169/) (2021)
- [Towards Exploiting Background Knowledge for Building Conversation Systems](https://mlsanity.com/paper/1809.08205/) (2018)
- [Low-Resource Knowledge-Grounded Dialogue Generation](https://mlsanity.com/paper/2002.10348/) (2020)
- [Almost Unsupervised Text to Speech and Automatic Speech Recognition](https://mlsanity.com/paper/1905.06791/) (2019)
- [Question Answering and Question Generation as Dual Tasks](https://mlsanity.com/paper/1706.02027/) (2017)
- [Zero-Resource Knowledge-Grounded Dialogue Generation](https://mlsanity.com/paper/2008.12918/) (2020)
- [Dual Supervised Learning](https://mlsanity.com/paper/1707.00415/) (2017)
- [Conditional Image-to-Image Translation](https://mlsanity.com/paper/1805.00251/) (2018)
- [A Document-grounded Matching Network for Response Selection in Retrieval-based Chatbots](https://mlsanity.com/paper/1906.04362/) (2019)

## Built on by (most cited first, 16 of 16)

- [Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization](https://mlsanity.com/paper/2406.01171/) (2024)
- [Lift Yourself Up: Retrieval-augmented Text Generation with Self Memory](https://mlsanity.com/paper/2305.02437/) (2023)
- [From Matching to Generation: A Survey on Generative Information Retrieval](https://mlsanity.com/paper/2404.14851/) (2024)
- [UniMS-RAG: A Unified Multi-source Retrieval-Augmented Generation for Personalized Dialogue Systems](https://mlsanity.com/paper/2401.13256/) (2024)
- [Bridging Cultural Nuances in Dialogue Agents through Cultural Value Surveys](https://mlsanity.com/paper/2401.10352/) (2024)
- [MallowsPO: Fine-Tune Your LLM with Preference Dispersions](https://mlsanity.com/paper/2405.14953/) (2024)
- [TPE: Towards Better Compositional Reasoning over Conceptual Tools with Multi-persona Collaboration](https://mlsanity.com/paper/2309.16090/) (2023)
- [MIRACLE: Towards Personalized Dialogue Generation with Latent-Space Multiple Personal Attribute Control](https://mlsanity.com/paper/2310.18342/) (2023)
- [WHAT, WHEN, and HOW to Ground: Designing User Persona-Aware Conversational Agents for Engaging Dialogue](https://mlsanity.com/paper/2306.03361/) (2023)
- [EZInterviewer: To Improve Job Interview Performance with Mock Interview Generator](https://mlsanity.com/paper/2301.00972/) (2023)
- [Diverse and Faithful Knowledge-Grounded Dialogue Generation via Sequential Posterior Inference](https://mlsanity.com/paper/2306.01153/) (2023)
- [Large Language Models as Source Planner for Personalized Knowledge-grounded Dialogue](https://mlsanity.com/paper/2310.08840/) (2023)
- [Improving Variational Autoencoders with Density Gap-based Regularization](https://mlsanity.com/paper/2211.00321/) (2022)
- [Learning to Express in Knowledge-Grounded Conversation](https://mlsanity.com/paper/2204.05805/) (2022)
- [Personalized LLM for Generating Customized Responses to the Same Query from Different Users](https://mlsanity.com/paper/2412.11736/) (2024)
- [PICK: Polished & Informed Candidate Scoring for Knowledge-Grounded Dialogue Systems](https://mlsanity.com/paper/2309.10413/) (2023)

## Abstract (arXiv)

Knowledge-grounded conversation (KGC) shows great potential in building an engaging and knowledgeable chatbot, and knowledge selection is a key ingredient in it. However, previous methods for knowledge selection only concentrate on the relevance between knowledge and dialogue context, ignoring the fact that age, hobby, education and life experience of an interlocutor have a major effect on his or her personal preference over external knowledge. Without taking the personalization issue into account, it is difficult to select the proper knowledge and generate persona-consistent responses. In this work, we introduce personal memory into knowledge selection in KGC to address the personalization issue. We propose a variational method to model the underlying relationship between one's personal memory and his or her selection of knowledge, and devise a learning scheme in which the forward mapping from personal memory to knowledge and its inverse mapping is included in a closed loop so that they could teach each other. Experiment results show that our method outperforms existing KGC methods significantly on both automatic evaluation and human evaluation.