# Building a Personalized Dialogue System with Prompt-Tuning

arXiv: https://arxiv.org/abs/2206.05399 · HTML: https://mlsanity.com/paper/2206.05399/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 23 references that are in the map.
- First posted on arXiv 11 Jun 2022.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [Personalizing Dialogue Agents: I have a dog, do you have pets too?](https://mlsanity.com/paper/1801.07243/) (2018)
- [GPT Understands, Too](https://mlsanity.com/paper/2103.10385/) (2021)
- [Multimodal Few-Shot Learning with Frozen Language Models](https://mlsanity.com/paper/2106.13884/) (2021)
- [DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset](https://mlsanity.com/paper/1710.03957/) (2017)
- [Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm](https://mlsanity.com/paper/2102.07350/) (2021)
- [Recipes for building an open-domain chatbot](https://mlsanity.com/paper/2004.13637/) (2020)
- [Learning How to Ask: Querying LMs with Mixtures of Soft Prompts](https://mlsanity.com/paper/2104.06599/) (2021)
- [SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer](https://mlsanity.com/paper/2110.07904/) (2021)
- [A Persona-Based Neural Conversation Model](https://mlsanity.com/paper/1603.06155/) (2016)
- [Training Millions of Personalized Dialogue Agents](https://mlsanity.com/paper/1809.01984/) (2018)
- [What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers](https://mlsanity.com/paper/2109.04650/) (2021)
- [BoB: BERT Over BERT for Training Persona-based Dialogue Models from Limited Personalized Data](https://mlsanity.com/paper/2106.06169/) (2021)
- [Personalized Dialogue Generation with Diversified Traits](https://mlsanity.com/paper/1901.09672/) (2019)
- [Empirical Analysis of Training Strategies of Transformer-based Japanese Chit-chat Systems](https://mlsanity.com/paper/2109.05217/) (2021)

## Built on by (most cited first, 9 of 9)

- [Hello Again! LLM-powered Personalized Agent for Long-term Dialogue](https://mlsanity.com/paper/2406.05925/) (2024)
- [Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedback](https://mlsanity.com/paper/2303.05453/) (2023)
- [Are LLMs Robust for Spoken Dialogues?](https://mlsanity.com/paper/2401.02297/) (2024)
- [Should We Fine-Tune or RAG? Evaluating Different Techniques to Adapt LLMs for Dialogue](https://mlsanity.com/paper/2406.06399/) (2024)
- [Response Generation in Longitudinal Dialogues: Which Knowledge Representation Helps?](https://mlsanity.com/paper/2305.15908/) (2023)
- [Beyond Discrete Personas: Personality Modeling Through Journal Intensive Conversations](https://mlsanity.com/paper/2412.11250/) (2024)
- [Prompt Learning to Mitigate Catastrophic Forgetting in Cross-lingual Transfer for Open-domain Dialogue Generation](https://mlsanity.com/paper/2305.07393/) (2023)
- [User-Specific Dialogue Generation with User Profile-Aware Pre-Training Model and Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2409.00887/) (2024)
- [HARMONI: Multimodal Personalization of Multi-User Human-Robot Interactions with LLMs](https://mlsanity.com/paper/2601.19839/) (2026)

## Abstract (arXiv)

Dialogue systems without consistent responses are not fascinating. In this study, we build a dialogue system that can respond based on a given character setting (persona) to bring consistency. Considering the trend of the rapidly increasing scale of language models, we propose an approach that uses prompt-tuning, which has low learning costs, on pre-trained large-scale language models. The results of automatic and manual evaluations in English and Japanese show that it is possible to build a dialogue system with more natural and personalized responses using less computational resources than fine-tuning.