# Personalized Dialogue Generation with Persona-Adaptive Attention

arXiv: https://arxiv.org/abs/2210.15088 · HTML: https://mlsanity.com/paper/2210.15088/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 22 references that are in the map.
- First posted on arXiv 27 Oct 2022.

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [Personalizing Dialogue Agents: I have a dog, do you have pets too?](https://mlsanity.com/paper/1801.07243/) (2018)
- [Recipes for building an open-domain chatbot](https://mlsanity.com/paper/2004.13637/) (2020)
- [The Pushshift Reddit Dataset](https://mlsanity.com/paper/2001.08435/) (2020)
- [Beyond Goldfish Memory: Long-Term Open-Domain Conversation](https://mlsanity.com/paper/2107.07567/) (2021)
- [TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents](https://mlsanity.com/paper/1901.08149/) (2019)
- [The Second Conversational Intelligence Challenge (ConvAI2)](https://mlsanity.com/paper/1902.00098/) (2019)
- [Dialogue Natural Language Inference](https://mlsanity.com/paper/1811.00671/) (2018)
- [You Impress Me: Dialogue Generation via Mutual Persona Perception](https://mlsanity.com/paper/2004.05388/) (2020)
- [BoB: BERT Over BERT for Training Persona-based Dialogue Models from Limited Personalized Data](https://mlsanity.com/paper/2106.06169/) (2021)
- [A Pre-training Based Personalized Dialogue Generation Model with Persona-sparse Data](https://mlsanity.com/paper/1911.04700/) (2019)
- [A Model-Agnostic Data Manipulation Method for Persona-based Dialogue Generation](https://mlsanity.com/paper/2204.09867/) (2022)
- [Dually Interactive Matching Network for Personalized Response Selection in Retrieval-Based Chatbots](https://mlsanity.com/paper/1908.05859/) (2019)
- [DeepFD: Automated Fault Diagnosis and Localization for Deep Learning Programs](https://mlsanity.com/paper/2205.01938/) (2022)

## Built on by (most cited first, 24 of 24)

- [Recent Trends in Personalized Dialogue Generation: A Review of Datasets, Methodologies, and Evaluations](https://mlsanity.com/paper/2405.17974/) (2024)
- [Learning Retrieval Augmentation for Personalized Dialogue Generation](https://mlsanity.com/paper/2406.18847/) (2024)
- [Selective Prompting Tuning for Personalized Conversations with LLMs](https://mlsanity.com/paper/2406.18187/) (2024)
- [Whose Personae? Synthetic Persona Experiments in LLM Research and Pathways to Transparency](https://mlsanity.com/paper/2512.00461/) (2025)
- [Visually-Aware Audio Captioning With Adaptive Audio-Visual Attention](https://mlsanity.com/paper/2210.16428/) (2022)
- ["In Dialogues We Learn": Towards Personalized Dialogue Without Pre-defined Profiles through In-Dialogue Learning](https://mlsanity.com/paper/2403.03102/) (2024)
- [A Desideratum for Conversational Agents: Capabilities, Challenges, and Future Directions](https://mlsanity.com/paper/2504.16939/) (2025)
- [Evaluating the Smooth Control of Attribute Intensity in Text Generation with LLMs](https://mlsanity.com/paper/2406.04460/) (2024)
- [LAVCap: LLM-based Audio-Visual Captioning using Optimal Transport](https://mlsanity.com/paper/2501.09291/) (2025)
- [Response Generation in Longitudinal Dialogues: Which Knowledge Representation Helps?](https://mlsanity.com/paper/2305.15908/) (2023)
- [User-Specific Dialogue Generation with User Profile-Aware Pre-Training Model and Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2409.00887/) (2024)
- [MCPDial: A Minecraft Persona-driven Dialogue Dataset](https://mlsanity.com/paper/2410.21627/) (2024)
- [Modeling and Optimizing User Preferences in AI Copilots: A Comprehensive Survey and Taxonomy](https://mlsanity.com/paper/2505.21907/) (2025)
- [Agent WARPP: Workflow Adherence via Runtime Parallel Personalization](https://mlsanity.com/paper/2507.19543/) (2025)
- [Personalized LLM for Generating Customized Responses to the Same Query from Different Users](https://mlsanity.com/paper/2412.11736/) (2024)
- [Exploiting contextual information to improve stance detection in informal political discourse with LLMs](https://mlsanity.com/paper/2602.04750/) (2026)
- [Crafting a Good Prompt or Providing Exemplary Dialogues? A Study of In-Context Learning for Persona-based Dialogue Generation](https://mlsanity.com/paper/2402.09954/) (2024)
- [From Persona to Person: Enhancing the Naturalness with Multiple Discourse Relations Graph Learning in Personalized Dialogue Generation](https://mlsanity.com/paper/2506.11557/) (2025)
- [Score Before You Speak: Improving Persona Consistency in Dialogue Generation using Response Quality Scores](https://mlsanity.com/paper/2508.06886/) (2025)
- [Persona-Aware Alignment Framework for Personalized Dialogue Generation](https://mlsanity.com/paper/2511.10215/) (2025)
- [HARMONI: Multimodal Personalization of Multi-User Human-Robot Interactions with LLMs](https://mlsanity.com/paper/2601.19839/) (2026)
- [GateRA: Token-Aware Modulation for Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2511.17582/) (2025)
- [Personality-Aware Reinforcement Learning for Persuasive Dialogue with LLM-Driven Simulation](https://mlsanity.com/paper/2601.06877/) (2026)
- [Persona Non Grata: LLM Persona-Driven Generations in MCQA are Unstable in Distinct Dimensions](https://mlsanity.com/paper/2607.00937/) (2026)

## Abstract (arXiv)

Persona-based dialogue systems aim to generate consistent responses based on historical context and predefined persona. Unlike conventional dialogue generation, the persona-based dialogue needs to consider both dialogue context and persona, posing a challenge for coherent training. Specifically, this requires a delicate weight balance between context and persona. To achieve that, in this paper, we propose an effective framework with Persona-Adaptive Attention (PAA), which adaptively integrates the weights from the persona and context information via our designed attention. In addition, a dynamic masking mechanism is applied to the PAA to not only drop redundant information in context and persona but also serve as a regularization mechanism to avoid overfitting. Experimental results demonstrate the superiority of the proposed PAA framework compared to the strong baselines in both automatic and human evaluation. Moreover, the proposed PAA approach can perform equivalently well in a low-resource regime compared to models trained in a full-data setting, which achieve a similar result with only 20% to 30% of data compared to the larger models trained in the full-data setting. To fully exploit the effectiveness of our design, we designed several variants for handling the weighted information in different ways, showing the necessity and sufficiency of our weighting and masking designs.