# A Pre-training Based Personalized Dialogue Generation Model with Persona-sparse Data

arXiv: https://arxiv.org/abs/1911.04700 · HTML: https://mlsanity.com/paper/1911.04700/

## Key facts

- Cited by 74 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Nov 2019.

## Built on by (most cited first, 30 of 74)

- [Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization](https://mlsanity.com/paper/2406.01171/) (2024)
- [CharacterEval: A Chinese Benchmark for Role-Playing Conversational Agent Evaluation](https://mlsanity.com/paper/2401.01275/) (2024)
- [A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Models](https://mlsanity.com/paper/2201.05337/) (2022)
- [Advances and Challenges in Conversational Recommender Systems: A Survey](https://mlsanity.com/paper/2101.09459/) (2021)
- [Personalized Prompt Learning for Explainable Recommendation](https://mlsanity.com/paper/2202.07371/) (2022)
- [BoB: BERT Over BERT for Training Persona-based Dialogue Models from Limited Personalized Data](https://mlsanity.com/paper/2106.06169/) (2021)
- [Cue-CoT: Chain-of-thought Prompting for Responding to In-depth Dialogue Questions with LLMs](https://mlsanity.com/paper/2305.11792/) (2023)
- [Personalized Transformer for Explainable Recommendation](https://mlsanity.com/paper/2105.11601/) (2021)
- [The Oscars of AI Theater: A Survey on Role-Playing with Language Models](https://mlsanity.com/paper/2407.11484/) (2024)
- [Are Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systems](https://mlsanity.com/paper/2310.05280/) (2023)
- [Writer-Defined AI Personas for On-Demand Feedback Generation](https://mlsanity.com/paper/2309.10433/) (2023)
- [MMChat: Multi-Modal Chat Dataset on Social Media](https://mlsanity.com/paper/2108.07154/) (2021)
- [Duplex Conversation: Towards Human-like Interaction in Spoken Dialogue Systems](https://mlsanity.com/paper/2205.15060/) (2022)
- [Rethinking and Refining the Distinct Metric](https://mlsanity.com/paper/2202.13587/) (2022)
- [Enhancing Persona Consistency for LLMs' Role-Playing using Persona-Aware Contrastive Learning](https://mlsanity.com/paper/2503.17662/) (2025)
- [CPED: A Large-Scale Chinese Personalized and Emotional Dialogue Dataset for Conversational AI](https://mlsanity.com/paper/2205.14727/) (2022)
- [Diversifying Dialog Generation via Adaptive Label Smoothing](https://mlsanity.com/paper/2105.14556/) (2021)
- [Personalized Dialogue Generation with Persona-Adaptive Attention](https://mlsanity.com/paper/2210.15088/) (2022)
- [Enhancing Personalized Dialogue Generation with Contrastive Latent Variables: Combining Sparse and Dense Persona](https://mlsanity.com/paper/2305.11482/) (2023)
- [MPCHAT: Towards Multimodal Persona-Grounded Conversation](https://mlsanity.com/paper/2305.17388/) (2023)
- [Exploring Prompt-based Few-shot Learning for Grounded Dialog Generation](https://mlsanity.com/paper/2109.06513/) (2021)
- [COSPLAY: Concept Set Guided Personalized Dialogue Generation Across Both Party Personas](https://mlsanity.com/paper/2205.00872/) (2022)
- [A Personalized Dialogue Generator with Implicit User Persona Detection](https://mlsanity.com/paper/2204.07372/) (2022)
- [IQA-EVAL: Automatic Evaluation of Human-Model Interactive Question Answering](https://mlsanity.com/paper/2408.13545/) (2024)
- [TVShowGuess: Character Comprehension in Stories as Speaker Guessing](https://mlsanity.com/paper/2204.07721/) (2022)
- [Improving Personality Consistency in Conversation by Persona Extending](https://mlsanity.com/paper/2208.10816/) (2022)
- [Learning to Memorize Entailment and Discourse Relations for Persona-Consistent Dialogues](https://mlsanity.com/paper/2301.04871/) (2023)
- [Large Language Models Meet Harry Potter: A Bilingual Dataset for Aligning Dialogue Agents with Characters](https://mlsanity.com/paper/2211.06869/) (2022)
- [PRODIGy: a PROfile-based DIalogue Generation dataset](https://mlsanity.com/paper/2311.05195/) (2023)
- [Persona-Guided Planning for Controlling the Protagonist's Persona in Story Generation](https://mlsanity.com/paper/2204.10703/) (2022)

## Abstract (arXiv)

Endowing dialogue systems with personas is essential to deliver more human-like conversations. However, this problem is still far from well explored due to the difficulties of both embodying personalities in natural languages and the persona sparsity issue observed in most dialogue corpora. This paper proposes a pre-training based personalized dialogue model that can generate coherent responses using persona-sparse dialogue data. In this method, a pre-trained language model is used to initialize an encoder and decoder, and personal attribute embeddings are devised to model richer dialogue contexts by encoding speakers' personas together with dialogue histories. Further, to incorporate the target persona in the decoding process and to balance its contribution, an attention routing structure is devised in the decoder to merge features extracted from the target persona and dialogue contexts using dynamically predicted weights. Our model can utilize persona-sparse dialogues in a unified manner during the training process, and can also control the amount of persona-related features to exhibit during the inference process. Both automatic and manual evaluation demonstrates that the proposed model outperforms state-of-the-art methods for generating more coherent and persona consistent responses with persona-sparse data.