# Strength Lies in Differences! Improving Strategy Planning for Non-collaborative Dialogues via Diversified User Simulation

arXiv: https://arxiv.org/abs/2403.06769 · HTML: https://mlsanity.com/paper/2403.06769/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 57 references that are in the map.
- First posted on arXiv 11 Mar 2024.

## Builds on (most cited first, 30 of 30)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Towards Measuring the Representation of Subjective Global Opinions in Language Models](https://mlsanity.com/paper/2306.16388/) (2023)
- [SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents](https://mlsanity.com/paper/2310.11667/) (2023)
- [Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs](https://mlsanity.com/paper/2210.13312/) (2022)
- [Deal or No Deal? End-to-End Learning for Negotiation Dialogues](https://mlsanity.com/paper/1706.05125/) (2017)
- [Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good](https://mlsanity.com/paper/1906.06725/) (2019)
- [Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedback](https://mlsanity.com/paper/2305.10142/) (2023)
- [Decoupling Strategy and Generation in Negotiation Dialogues](https://mlsanity.com/paper/1808.09637/) (2018)
- [Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Game](https://mlsanity.com/paper/2310.18940/) (2023)
- [Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification, Target-guided, and Non-collaboration](https://mlsanity.com/paper/2305.13626/) (2023)
- [How Well Can LLMs Negotiate? NegotiationArena Platform and Analysis](https://mlsanity.com/paper/2402.05863/) (2024)
- [Plug-and-Play Policy Planner for Large Language Model Powered Dialogue Agents](https://mlsanity.com/paper/2311.00262/) (2023)
- [How FaR Are Large Language Models From Agents with Theory-of-Mind?](https://mlsanity.com/paper/2310.03051/) (2023)
- [A Survey on Proactive Dialogue Systems: Problems, Methods, and Prospects](https://mlsanity.com/paper/2305.02750/) (2023)
- [Boosting Theory-of-Mind Performance in Large Language Models via Prompting](https://mlsanity.com/paper/2304.11490/) (2023)
- [Prompt-Based Monte-Carlo Tree Search for Goal-Oriented Dialogue Policy Planning](https://mlsanity.com/paper/2305.13660/) (2023)
- [How to Build User Simulators to Train RL-based Dialog Systems](https://mlsanity.com/paper/1909.01388/) (2019)
- [MIC: Mining Interclass Characteristics for Improved Metric Learning](https://mlsanity.com/paper/1909.11574/) (2019)
- [Improving Dialog Systems for Negotiation with Personality Modeling](https://mlsanity.com/paper/2010.09954/) (2020)
- [Controllable Mixed-Initiative Dialogue Generation through Prompting](https://mlsanity.com/paper/2305.04147/) (2023)
- [Let's Negotiate! A Survey of Negotiation Dialogue Systems](https://mlsanity.com/paper/2212.09072/) (2022)
- [Ask an Expert: Leveraging Language Models to Improve Strategic Reasoning in Goal-Oriented Dialogue Models](https://mlsanity.com/paper/2305.17878/) (2023)
- [Are LLMs Effective Negotiators? Systematic Evaluation of the Multifaceted Capabilities of LLMs in Negotiation Dialogues](https://mlsanity.com/paper/2402.13550/) (2024)
- [RESPER: Computationally Modelling Resisting Strategies in Persuasive Conversations](https://mlsanity.com/paper/2101.10545/) (2021)
- [Interacting with Non-Cooperative User: A New Paradigm for Proactive Dialogue Policy](https://mlsanity.com/paper/2204.07433/) (2022)
- [Be Selfish, But Wisely: Investigating the Impact of Agent Personality in Mixed-Motive Human-Agent Interactions](https://mlsanity.com/paper/2310.14404/) (2023)
- [Social Influence Dialogue Systems: A Survey of Datasets and Models For Social Influence Tasks](https://mlsanity.com/paper/2210.05664/) (2022)
- [LEGOEval: An Open-Source Toolkit for Dialogue System Evaluation via Crowdsourcing](https://mlsanity.com/paper/2105.01992/) (2021)
- [Enhancing Large Language Model Induced Task-Oriented Dialogue Systems Through Look-Forward Motivated Goals](https://mlsanity.com/paper/2309.08949/) (2023)

## Built on by (most cited first, 21 of 21)

- [LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Models](https://mlsanity.com/paper/2404.01230/) (2024)
- [Towards Human-centered Proactive Conversational Agents](https://mlsanity.com/paper/2404.12670/) (2024)
- [Know You First and Be You Better: Modeling Human-Like User Simulators via Implicit Profiles](https://mlsanity.com/paper/2502.18968/) (2025)
- [Large Language Models for Planning: A Comprehensive and Systematic Survey](https://mlsanity.com/paper/2505.19683/) (2025)
- [SOTOPIA-Omega: Dynamic Strategy Injection Learning and Social Instruction Following Evaluation for Social Agents](https://mlsanity.com/paper/2502.15538/) (2025)
- [PRINCIPLES: Synthetic Strategy Memory for Proactive Dialogue Agents](https://mlsanity.com/paper/2509.17459/) (2025)
- [ProactiveEval: A Unified Evaluation Framework for Proactive Dialogue Agents](https://mlsanity.com/paper/2508.20973/) (2025)
- [Simulating Before Planning: Constructing Intrinsic User World Model for User-Tailored Dialogue Policy Planning](https://mlsanity.com/paper/2504.13643/) (2025)
- [How to Enable Effective Cooperation Between Humans and NLP Models: A Survey of Principles, Formalizations, and Beyond](https://mlsanity.com/paper/2501.05714/) (2025)
- [DialogXpert: Driving Intelligent and Emotion-Aware Conversations through Online Value-Based Reinforcement Learning with LLM Priors](https://mlsanity.com/paper/2505.17795/) (2025)
- [METRO: Towards Strategy Induction from Expert Dialogue Transcripts for Non-collaborative Dialogues](https://mlsanity.com/paper/2604.11427/) (2026)
- [Beyond Static Personas: Situational Personality Steering for Large Language Models](https://mlsanity.com/paper/2604.13846/) (2026)
- [Dual Hierarchical Dialogue Policy Learning for Legal Inquisitive Conversational Agents](https://mlsanity.com/paper/2605.14057/) (2026)
- [A General Highly Accurate Online Planning Method Integrating Large Language Models into Nested Rollout Policy Adaptation for Dialogue Tasks](https://mlsanity.com/paper/2511.21706/) (2025)
- [Enhancing Persuasive Dialogue Agents by Synthesizing Cross-Disciplinary Communication Strategies](https://mlsanity.com/paper/2602.22696/) (2026)
- [Effects of Theory of Mind and Prosocial Beliefs on Steering Human-Aligned Behaviors of LLMs in Ultimatum Games](https://mlsanity.com/paper/2505.24255/) (2025)
- [CoSToM:Causal-oriented Steering for Intrinsic Theory-of-Mind Alignment in Large Language Models](https://mlsanity.com/paper/2604.10031/) (2026)
- [UP-NRPA: User Portrait based Nested Rollout Policy Adaptation for Planning with Large Language Models in Goal-oriented Dialogue Systems](https://mlsanity.com/paper/2606.13683/) (2026)
- [Clarify User Expertise: Towards Proactive Conversational Agents Tailoring Responses to User Proficiency](https://mlsanity.com/paper/2608.22266/) (2026)
- [Towards Effective Structured Context Modeling for Conversational Recommender Systems via Dual-node Monte Carlo Tree Search](https://mlsanity.com/paper/2609.00618/) (2026)
- [Proactive Dialogue Policy Optimization via Cognitive-State Transition](https://mlsanity.com/paper/2609.34948/) (2026)

## Abstract (arXiv)

We investigate non-collaborative dialogue agents, which are expected to engage in strategic conversations with diverse users, for securing a mutual agreement that leans favorably towards the system's objectives. This poses two main challenges for existing dialogue agents: 1) The inability to integrate user-specific characteristics into the strategic planning, and 2) The difficulty of training strategic planners that can be generalized to diverse users. To address these challenges, we propose Trip to enhance the capability in tailored strategic planning, incorporating a user-aware strategic planning module and a population-based training paradigm. Through experiments on benchmark non-collaborative dialogue tasks, we demonstrate the effectiveness of Trip in catering to diverse users.