Towards Zero-Shot, Controllable Dialog Planning with LLMs
On the map: LLM reasoning → Reasoning for agents, planning and code
Abstract (arXiv)
Recently, Large Language Models (LLMs) have emerged as an alternative to training task-specific dialog agents, due to their broad reasoning capabilities and performance in zero-shot learning scenarios. However, many LLM-based dialog systems fall short in planning towards an overarching dialog goal and therefore cannot steer the conversation appropriately. Furthermore, these models struggle with hallucination, making them unsuitable for information access in sensitive domains, such as legal or medical domains, where correctness of information given to users is critical. The recently introduced task Conversational Tree Search (CTS) proposes the use of dialog graphs to avoid hallucination in sensitive domains, however, state-of-the-art agents are Reinforcement Learning (RL) based and require long training times, despite excelling at dialog strategy. This paper introduces a novel zero-shot method for controllable CTS agents, where LLMs guide the dialog planning through domain graphs by searching and pruning relevant graph nodes based on user interaction preferences. We show that these agents significantly outperform state-of-the-art CTS agents (p<0.0001; Barnard Exact test) in simulation. This generalizes to all available CTS domains. Finally, we perform user evaluation to test the agent's performance in the wild, showing that our policy significantly (p<0.05; Barnard Exact) improves task-success compared to the state-of-the-art RL-based CTS agent.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 15 of its 44 references that are in the map. First posted on arXiv 8 Oct 2024.
Builds on
15 of 44 references.
References · 15
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
- 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
- 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
- 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
- 2022Towards Reasoning in Large Language Models: A Surveycited by 593
- 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
- 2023Plug-and-Play Policy Planner for Large Language Model Powered Dialogue Agentscited by 76
- 2021CINS: Comprehensive Instruction for Few-shot Learning in Task-oriented Dialog Systemscited by 30
- 2022A Survey on Recent Advances and Challenges in Reinforcement Learning Methods for Task-Oriented Dialogue Policy Learningcited by 30
- 2020Language Model is All You Need: Natural Language Understanding as Question Answeringcited by 29
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024ChatSOP: An SOP-Guided MCTS Planning Framework for Controllable LLM Dialogue Agents
- 2025The Behavior Gap: Evaluating Zero-shot LLM Agents in Complex Task-Oriented Dialogs
- 2025SAGE: Steering Dialog Generation with Future-Aware State-Action Augmentation
- 2025SpeakRL: Synergizing Reasoning, Speaking, and Acting in Language Models with Reinforcement Learning
- 2022A Mixture-of-Expert Approach to RL-based Dialogue Management
- 2026StateTree: Enhancing Long-Term Dialogue Reasoning via Reinforcement Learning
- 2026Agentic Conversational Search with Contextualized Reasoning via Reinforcement Learning
- 2025Conversational Planning for Personal Plans