# Refine and Imitate: Reducing Repetition and Inconsistency in Persuasion Dialogues via Reinforcement Learning and Human Demonstration

arXiv: https://arxiv.org/abs/2012.15375 · HTML: https://mlsanity.com/paper/2012.15375/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Dec 2020.

## Built on by (most cited first, 20 of 20)

- [A Survey on Proactive Dialogue Systems: Problems, Methods, and Prospects](https://mlsanity.com/paper/2305.02750/) (2023)
- [Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language Generation](https://mlsanity.com/paper/2305.00955/) (2023)
- [Let's Negotiate! A Survey of Negotiation Dialogue Systems](https://mlsanity.com/paper/2212.09072/) (2022)
- [COOPER: Coordinating Specialized Agents towards a Complex Dialogue Goal](https://mlsanity.com/paper/2312.11792/) (2023)
- [Exploring the Potential of LLMs as Personalized Assistants: Dataset, Evaluation, and Analysis](https://mlsanity.com/paper/2506.01262/) (2025)
- [Leveraging Large Language Models for Automated Dialogue Analysis](https://mlsanity.com/paper/2309.06490/) (2023)
- [Building Persona Consistent Dialogue Agents with Offline Reinforcement Learning](https://mlsanity.com/paper/2310.10735/) (2023)
- [Zero-shot Persuasive Chatbots with LLM-Generated Strategies and Information Retrieval](https://mlsanity.com/paper/2407.03585/) (2024)
- [Must Read: A Comprehensive Survey of Computational Persuasion](https://mlsanity.com/paper/2505.07775/) (2025)
- [ToMAP: Training Opponent-Aware LLM Persuaders with Theory of Mind](https://mlsanity.com/paper/2505.22961/) (2025)
- [Seamlessly Integrating Factual Information and Social Content with Persuasive Dialogue](https://mlsanity.com/paper/2203.07657/) (2022)
- [Preemptive Detection and Correction of Misaligned Actions in LLM Agents](https://mlsanity.com/paper/2407.11843/) (2024)
- [Evaluating Task-oriented Dialogue Systems: A Systematic Review of Measures, Constructs and their Operationalisations](https://mlsanity.com/paper/2312.13871/) (2023)
- [Affective Multimodal Agents with Proactive Knowledge Grounding for Emotionally Aligned Marketing Dialogue](https://mlsanity.com/paper/2511.21728/) (2025)
- [Harnessing Knowledge and Reasoning for Human-Like Natural Language Generation: A Brief Review](https://mlsanity.com/paper/2212.03747/) (2022)
- [AutoReply: Detecting Nonsense in Dialogue Introspectively with Discriminative Replies](https://mlsanity.com/paper/2211.12615/) (2022)
- [MADS: Multi-Agent Dialogue Simulation for Diverse Persuasion Data Generation](https://mlsanity.com/paper/2510.05124/) (2025)
- [A Review of Reinforcement Learning for Natural Language Processing, and Applications in Healthcare](https://mlsanity.com/paper/2310.18354/) (2023)
- [One Model, Multiple Goals: Adaptive Multi-Objective Learning for E-commerce Dialogue Systems](https://mlsanity.com/paper/2606.09293/) (2026)
- [Let's Negotiate! A Survey of Negotiation Dialogue Systems](https://mlsanity.com/paper/2402.01097/) (2024)

## Abstract (arXiv)

Persuasion dialogue systems reflect the machine's ability to make strategic moves beyond verbal communication, and therefore differentiate themselves from task-oriented or open-domain dialogue systems and have their own unique values. However, the repetition and inconsistency problems still persist in dialogue response generation and could substantially impact user experience and impede the persuasion outcome. Besides, although reinforcement learning (RL) approaches have achieved big success in strategic tasks such as games, they require a sophisticated user simulator to provide real-time feedback to the dialogue system, which limits the application of RL on persuasion dialogues. To address these issues towards a better persuasion dialogue system, we apply RL to refine a language model baseline without user simulators, and distill sentence-level information about repetition, inconsistency, and task relevance through rewards. Moreover, to better accomplish the persuasion task, the model learns from human demonstration to imitate human persuasion behavior and selects the most persuasive responses. Experiments show that our model outperforms previous state-of-the-art dialogue models on both automatic metrics and human evaluation results on a donation persuasion task, and generates more diverse, consistent and persuasive conversations according to the user feedback.