mlsanityData to 7 Oct 2026

Planning Like Human: A Dual-process Framework for Dialogue Planning

arXiv 2406.05374 ↗First posted 8 Jun 2024cs.CLcited by 32

Not on the map. Why?

Abstract (arXiv)

In proactive dialogue, the challenge lies not just in generating responses but in steering conversations toward predetermined goals, a task where Large Language Models (LLMs) typically struggle due to their reactive nature. Traditional approaches to enhance dialogue planning in LLMs, ranging from elaborate prompt engineering to the integration of policy networks, either face efficiency issues or deliver suboptimal performance. Inspired by the dualprocess theory in psychology, which identifies two distinct modes of thinking - intuitive (fast) and analytical (slow), we propose the Dual-Process Dialogue Planning (DPDP) framework. DPDP embodies this theory through two complementary planning systems: an instinctive policy model for familiar contexts and a deliberative Monte Carlo Tree Search (MCTS) mechanism for complex, novel scenarios. This dual strategy is further coupled with a novel two-stage training regimen: offline Reinforcement Learning for robust initial policy model formation followed by MCTS-enhanced on-the-fly learning, which ensures a dynamic balance between efficiency and strategic depth. Our empirical evaluations across diverse dialogue tasks affirm DPDP's superiority in achieving both high-quality dialogues and operational efficiency, outpacing existing methods.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 39 of its 58 references that are in the map. First posted on arXiv 8 Jun 2024.

Builds on

39 of 58 references.

References · 39

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  6. 2020Learning to summarize from human feedbackcited by 2,562
  7. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  8. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  9. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 849
  10. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  11. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 723
  12. 2021Towards Emotional Support Dialog Systemscited by 294
27 more
  1. 2017Thinking Fast and Slow with Deep Learning and Tree Searchcited by 245
  2. 2023Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedbackcited by 202
  3. 2023AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotatorscited by 171
  4. 2018Decoupling Strategy and Generation in Negotiation Dialoguescited by 156
  5. 2023Large Language Model Is Not a Good Few-shot Information Extractor, but a Good Reranker for Hard Samples!cited by 119
  6. 2023Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification, Target-guided, and Non-collaborationcited by 115
  7. 2023Retroformer: Retrospective Large Language Agents with Policy Gradient Optimizationcited by 110
  8. 2017The Consciousness Priorcited by 105
  9. 2023Guiding Large Language Models via Directional Stimulus Promptingcited by 94
  10. 2023Plug-and-Play Policy Planner for Large Language Model Powered Dialogue Agentscited by 76
  11. 2023Prompt-Based Monte-Carlo Tree Search for Goal-Oriented Dialogue Policy Planningcited by 60
  12. 2022Improving Multi-turn Emotional Support Dialogue Generation with Lookahead Strategy Planningcited by 59
  13. 2018Dialogue Learning with Human Teaching and Feedback in End-to-End Trainable Task-Oriented Dialogue Systemscited by 53
  14. 2023SGP-TOD: Building Task Bots Effortlessly via Schema-Guided LLM Promptingcited by 39
  15. 2023Ask an Expert: Leveraging Language Models to Improve Strategic Reasoning in Goal-Oriented Dialogue Modelscited by 30
  16. 2021DialoGraph: Incorporating Interpretable Strategy-Graph Networks into Negotiation Dialoguescited by 25
  17. 2020Multi-Agent Task-Oriented Dialog Policy Learning with Role-Aware Reward Decompositioncited by 23
  18. 2023Fantastic Rewards and How to Tame Them: A Case Study on Reward Learning for Task-oriented Dialogue Systemscited by 19
  19. 2018How to Combine Tree-Search Methods in Reinforcement Learningcited by 16
  20. 2020Learning Goal-oriented Dialogue Policy with Opposite Agent Awarenesscited by 16
  21. 2023Dialogue Planning via Brownian Bridge Stochastic Process for Goal-directed Proactive Dialoguecited by 14
  22. 2016Deep Learning for Reward Design to Improve Monte Carlo Tree Search in ATARI Gamescited by 13
  23. 2019Deep Reasoning Networks: Thinking Fast and Slowcited by 12
  24. 2019Policy Gradient Search: Online Planning and Expert Iteration without Search Treescited by 11
  25. 2017Thinking, Fast and Slow: Combining Vector Spaces and Knowledge Graphscited by 10
  26. 2023Conversational Tree Search: A New Hybrid Dialog Taskcited by 7
  27. 2019Learning Policies from Self-Play with Policy Gradients and MCTS Value Estimatescited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 17; 2026: 10.

32 papers.

Sort:

By area:

  1. 2025Adaptive Social Learning via Mode Policy Optimization for Language Agentscited by 15
  2. 2025PRINCIPLES: Synthetic Strategy Memory for Proactive Dialogue Agentscited by 15
  3. 2024Simulation-Free Hierarchical Latent Policy Planning for Proactive Dialoguescited by 14
  4. 2025Reasoning on a Spectrum: Aligning LLMs to System 1 and System 2 Thinkingcited by 14
  5. 2024ChatSOP: An SOP-Guided MCTS Planning Framework for Controllable LLM Dialogue Agentscited by 11
  6. 2024The Imperative of Conversation Analysis in the Era of LLMs: A Survey of Tasks, Techniques, and Trendscited by 11
  7. 2025Leveraging Dual Process Theory in Language Agent Framework for Real-time Simultaneous Human-AI Collaborationcited by 10
  8. 2025Debate, Reflect, and Distill: Multi-Agent Feedback with Tree-Structured Preference Optimization for Efficient Language Model Enhancementcited by 10
  9. 2025Simulating Before Planning: Constructing Intrinsic User World Model for User-Tailored Dialogue Policy Planningcited by 7
  10. 2024PCQPR: Proactive Conversational Question Planning with Reflectioncited by 6
  11. 2024SAPIENT: Mastering Multi-turn Conversational Recommendation with Strategic Planning and Monte Carlo Tree Searchcited by 5
  12. 2025DecoupledESC: Enhancing Emotional Support Generation via Strategy-Response Decoupled Preference Optimizationcited by 5
20 more
  1. 2025Towards Open-Ended Emotional Support Conversations in LLMs via Reinforcement Learning with Future-Oriented Rewardscited by 5
  2. 2025DialogXpert: Driving Intelligent and Emotion-Aware Conversations through Online Value-Based Reinforcement Learning with LLM Priorscited by 4
  3. 2025DiaCBT: A Long-Periodic Dialogue Corpus Guided by Cognitive Conceptualization Diagram for CBT-based Psychological Counselingcited by 4
  4. 2026PATS: Personality-Aware Teaching Strategies with Large Language Model Tutorscited by 4
  5. 2025Bridging Social Psychology and LLM Reasoning: Conflict-Aware Meta-Review Generation via Cognitive Alignmentcited by 2
  6. 2025Toward Real-World Chinese Psychological Support Dialogues: CPsDD Dataset and a Co-Evolving Multi-Agent Systemcited by 2
  7. 2026From Myopic Selection to Long-Horizon Awareness: Sequential LLM Routing for Multi-Turn Dialoguecited by 2
  8. 2026Proactive Service Agents: A Unified Decision Framework, Methods, and Evaluationcited by 2
  9. 2025Tailored Conversations beyond LLMs: A RL-Based Dialogue Managercited by 1
  10. 2025CogniPlay: a work-in-progress Human-like model for General Game Playingcited by 1
  11. 2025A General Highly Accurate Online Planning Method Integrating Large Language Models into Nested Rollout Policy Adaptation for Dialogue Taskscited by 1
  12. 2026Can large language models interpret unstructured chat data on dynamic group decision-making processes? Evidence on joint destination choicecited by 1
  13. 2026Enhancing Persuasive Dialogue Agents by Synthesizing Cross-Disciplinary Communication Strategiescited by 1
  14. 2025Planning with Diffusion Models for Target-Oriented Dialogue Systemscited by 0
  15. 2025DyBBT: Dynamic Balance via Bandit-inspired Targeting for Dialog Policy with Cognitive Dual-Systemscited by 0
  16. 2026Knowledge-informed Bidding with Dual-process Control for Online Advertisingcited by 0
  17. 2026UP-NRPA: User Portrait based Nested Rollout Policy Adaptation for Planning with Large Language Models in Goal-oriented Dialogue Systemscited by 0
  18. 2026Everyone is unique: Towards Behaviorally Heterogeneous Negotiation Dialogue Systems for Debt Collectioncited by 0
  19. 2026STRETCH the Boundaries: A Unified Self-Taught Framework for Progressive LLM Evolutioncited by 0
  20. 2026Proactive Dialogue Policy Optimization via Cognitive-State Transitioncited by 0