# Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification, Target-guided, and Non-collaboration

arXiv: https://arxiv.org/abs/2305.13626 · HTML: https://mlsanity.com/paper/2305.13626/

## Key facts

- Cited by 116 papers in the map (data through 14 Oct 2026).
- Builds on 57 of its 76 references that are in the map.
- First posted on arXiv 23 May 2023.

## Builds on (most cited first, 30 of 57)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents](https://mlsanity.com/paper/2201.07207/) (2022)
- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://mlsanity.com/paper/2302.04023/) (2023)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [CoQA: A Conversational Question Answering Challenge](https://mlsanity.com/paper/1808.07042/) (2018)
- [Recipes for building an open-domain chatbot](https://mlsanity.com/paper/2004.13637/) (2020)
- [Towards a Human-like Open-Domain Chatbot](https://mlsanity.com/paper/2001.09977/) (2020)
- [TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance](https://mlsanity.com/paper/2105.07624/) (2021)
- [ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPT](https://mlsanity.com/paper/2302.10205/) (2023)
- [Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good](https://mlsanity.com/paper/1906.06725/) (2019)
- [Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedback](https://mlsanity.com/paper/2305.10142/) (2023)
- [How Many Data Points is a Prompt Worth?](https://mlsanity.com/paper/2103.08493/) (2021)
- [TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue](https://mlsanity.com/paper/2004.06871/) (2020)
- [The Second Conversational Intelligence Challenge (ConvAI2)](https://mlsanity.com/paper/1902.00098/) (2019)
- [Decoupling Strategy and Generation in Negotiation Dialogues](https://mlsanity.com/paper/1808.09637/) (2018)
- [Language Models that Seek for Knowledge: Modular Search & Generation for Dialogue and Prompt Completion](https://mlsanity.com/paper/2203.13224/) (2022)
- [Towards Topic-Guided Conversational Recommender System](https://mlsanity.com/paper/2010.04125/) (2020)
- [Dialogue State Tracking with a Language Model using Schema-Driven Prompting](https://mlsanity.com/paper/2109.07506/) (2021)
- [Proactive Human-Machine Conversation with Explicit Conversation Goals](https://mlsanity.com/paper/1906.05572/) (2019)
- [Recent Advances and Challenges in Task-oriented Dialog System](https://mlsanity.com/paper/2003.07490/) (2020)
- [PLACES: Prompting Language Models for Social Conversation Synthesis](https://mlsanity.com/paper/2302.03269/) (2023)

## Built on by (most cited first, 30 of 116)

- [CollabLLM: From Passive Responders to Active Collaborators](https://mlsanity.com/paper/2502.00640/) (2025)
- [Plug-and-Play Policy Planner for Large Language Model Powered Dialogue Agents](https://mlsanity.com/paper/2311.00262/) (2023)
- [Modeling Future Conversation Turns to Teach LLMs to Ask Clarifying Questions](https://mlsanity.com/paper/2410.13788/) (2024)
- [Building Emotional Support Chatbots in the Era of LLMs](https://mlsanity.com/paper/2308.11584/) (2023)
- [Ask-before-Plan: Proactive Language Agents for Real-World Planning](https://mlsanity.com/paper/2406.12639/) (2024)
- [Learning to Clarify: Multi-turn Conversations with Action-Based Contrastive Self-Training](https://mlsanity.com/paper/2406.00222/) (2024)
- [CLAMBER: A Benchmark of Identifying and Clarifying Ambiguous Information Needs in Large Language Models](https://mlsanity.com/paper/2405.12063/) (2024)
- [Towards Human-centered Proactive Conversational Agents](https://mlsanity.com/paper/2404.12670/) (2024)
- [Structured Uncertainty guided Clarification for LLM Agents](https://mlsanity.com/paper/2511.08798/) (2025)
- [Knowledge Boundary of Large Language Models: A Survey](https://mlsanity.com/paper/2412.12472/) (2024)
- [Asking the Right Question at the Right Time: Human and Model Uncertainty Guidance to Ask Clarification Questions](https://mlsanity.com/paper/2402.06509/) (2024)
- [Planning Like Human: A Dual-process Framework for Dialogue Planning](https://mlsanity.com/paper/2406.05374/) (2024)
- [Don't Just Say "I don't know"! Self-aligning Large Language Models for Responding to Unknown Questions with Explanations](https://mlsanity.com/paper/2402.15062/) (2024)
- [WatME: Towards Lossless Watermarking Through Lexical Redundancy](https://mlsanity.com/paper/2311.09832/) (2023)
- [Affective Computing in the Era of Large Language Models: A Survey from the NLP Perspective](https://mlsanity.com/paper/2408.04638/) (2024)
- [A Survey on Recent Advances in Conversational Data Generation](https://mlsanity.com/paper/2405.13003/) (2024)
- [Strength Lies in Differences! Improving Strategy Planning for Non-collaborative Dialogues via Diversified User Simulation](https://mlsanity.com/paper/2403.06769/) (2024)
- [PRACTIQ: A Practical Conversational Text-to-SQL dataset with Ambiguous and Unanswerable Queries](https://mlsanity.com/paper/2410.11076/) (2024)
- [COOPER: Coordinating Specialized Agents towards a Complex Dialogue Goal](https://mlsanity.com/paper/2312.11792/) (2023)
- [STYLE: Improving Domain Transferability of Asking Clarification Questions in Large Language Model Powered Conversational Agents](https://mlsanity.com/paper/2405.12059/) (2024)
- [PRINCIPLES: Synthetic Strategy Memory for Proactive Dialogue Agents](https://mlsanity.com/paper/2509.17459/) (2025)
- [Depression Diagnosis Dialogue Simulation: Self-improving Psychiatrist with Tertiary Memory](https://mlsanity.com/paper/2409.15084/) (2024)
- [Simulation-Free Hierarchical Latent Policy Planning for Proactive Dialogues](https://mlsanity.com/paper/2412.14584/) (2024)
- [Consistent Client Simulation for Motivational Interviewing-based Counseling](https://mlsanity.com/paper/2502.02802/) (2025)
- [ClarifyMT-Bench: Benchmarking and Improving Multi-Turn Clarification for Conversational Large Language Models](https://mlsanity.com/paper/2512.21120/) (2025)
- [Value of Information: A Framework for Human-Agent Communication](https://mlsanity.com/paper/2601.06407/) (2026)
- [Think Before You Speak: Cultivating Communication Skills of Large Language Models via Inner Monologue](https://mlsanity.com/paper/2311.07445/) (2023)
- [Instruct Once, Chat Consistently in Multiple Rounds: An Efficient Tuning Framework for Dialogue](https://mlsanity.com/paper/2402.06967/) (2024)
- [The Imperative of Conversation Analysis in the Era of LLMs: A Survey of Tasks, Techniques, and Trends](https://mlsanity.com/paper/2409.14195/) (2024)
- [Teaching Vision-Language Models to Ask: Resolving Ambiguity in Visual Questions](https://mlsanity.com/paper/2507.13773/) (2025)

## Abstract (arXiv)

Conversational systems based on Large Language Models (LLMs), such as ChatGPT, show exceptional proficiency in context understanding and response generation. However, despite their impressive capabilities, they still possess limitations, such as providing randomly-guessed answers to ambiguous queries or failing to refuse users' requests, both of which are considered aspects of a conversational agent's proactivity. This raises the question of whether LLM-based conversational systems are equipped to handle proactive dialogue problems. In this work, we conduct a comprehensive analysis of LLM-based conversational systems, specifically focusing on three aspects of proactive dialogue systems: clarification, target-guided, and non-collaborative dialogues. To trigger the proactivity of LLMs, we propose the Proactive Chain-of-Thought prompting scheme, which augments LLMs with the goal planning capability over descriptive reasoning chains. Empirical findings are discussed to promote future studies on LLM-based proactive dialogue systems.