# A Survey on Asking Clarification Questions Datasets in Conversational Systems

arXiv: https://arxiv.org/abs/2305.15933 · HTML: https://mlsanity.com/paper/2305.15933/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 May 2023.

## Built on by (most cited first, 28 of 28)

- [Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Models](https://mlsanity.com/paper/2402.14207/) (2024)
- [CollabLLM: From Passive Responders to Active Collaborators](https://mlsanity.com/paper/2502.00640/) (2025)
- [CLAMBER: A Benchmark of Identifying and Clarifying Ambiguous Information Needs in Large Language Models](https://mlsanity.com/paper/2405.12063/) (2024)
- [Grounding Gaps in Language Model Generations](https://mlsanity.com/paper/2311.09144/) (2023)
- [AMBROSIA: A Benchmark for Parsing Ambiguous Questions into Database Queries](https://mlsanity.com/paper/2406.19073/) (2024)
- [Navigating Rifts in Human-LLM Grounding: Study and Benchmark](https://mlsanity.com/paper/2503.13975/) (2025)
- [STYLE: Improving Domain Transferability of Asking Clarification Questions in Large Language Model Powered Conversational Agents](https://mlsanity.com/paper/2405.12059/) (2024)
- [Anthropomimetic Uncertainty: What Verbalized Uncertainty in Language Models is Missing](https://mlsanity.com/paper/2507.10587/) (2025)
- [HumanAgencyBench: Scalable Evaluation of Human Agency Support in AI Assistants](https://mlsanity.com/paper/2509.08494/) (2025)
- [ProductAgent: Benchmarking Conversational Product Search Agent with Asking Clarification Questions](https://mlsanity.com/paper/2407.00942/) (2024)
- [InfoQuest: Evaluating Multi-Turn Dialogue Agents for Open-Ended Conversations with Hidden Context](https://mlsanity.com/paper/2502.12257/) (2025)
- [Dynamic Contexts for Generating Suggestion Questions in RAG Based Conversational Systems](https://mlsanity.com/paper/2403.11413/) (2024)
- [Towards Personalized Conversational Sales Agents: Contextual User Profiling for Strategic Action](https://mlsanity.com/paper/2504.08754/) (2025)
- [Taking Action Towards Graceful Interaction: The Effects of Performing Actions on Modelling Policies for Instruction Clarification Requests](https://mlsanity.com/paper/2401.17039/) (2024)
- [Curiosity by Design: An LLM-based Coding Assistant Asking Clarification Questions](https://mlsanity.com/paper/2507.21285/) (2025)
- [IDRBench: Benchmarking the Interactive Capabilities of Deep Research Agents](https://mlsanity.com/paper/2601.06676/) (2026)
- [Synthetic Clarification and Correction Dialogues about Data-Centric Tasks -- A Teacher-Student Approach](https://mlsanity.com/paper/2503.14167/) (2025)
- [CLEAR-KGQA: Clarification-Enhanced Ambiguity Resolution for Knowledge Graph Question Answering](https://mlsanity.com/paper/2504.09665/) (2025)
- [Alexpaca: Learning Factual Clarification Question Generation Without Examples](https://mlsanity.com/paper/2310.11571/) (2023)
- [What Users Leave Unsaid: Under-Specified Queries Limit Vision-Language Models](https://mlsanity.com/paper/2601.06165/) (2026)
- [Generating Clarification Questions for Disambiguating Contracts](https://mlsanity.com/paper/2403.08053/) (2024)
- [Correct-Detect: Balancing Performance and Ambiguity Through the Lens of Coreference Resolution in LLMs](https://mlsanity.com/paper/2509.14456/) (2025)
- [From Answers to Guidance: A Proactive Dialogue System for Legal Documents](https://mlsanity.com/paper/2510.19723/) (2025)
- [WASIL: In-the-Wild Arabic Spoken Interactions with LLMs](https://mlsanity.com/paper/2605.16364/) (2026)
- [Lexical Entrainment for Conversational Systems](https://mlsanity.com/paper/2310.09651/) (2023)
- [Dagstuhl Perspectives Workshop 24352 -- Conversational Agents: A Framework for Evaluation (CAFE): Manifesto](https://mlsanity.com/paper/2506.11112/) (2025)
- [Knowing When to Ask: Self-Gated Clarification for Hierarchical Language Agents](https://mlsanity.com/paper/2606.11349/) (2026)
- [Turning Intent into Specifications: A Benchmark and an Interactive User-Assistant Agent](https://mlsanity.com/paper/2606.20585/) (2026)

## Abstract (arXiv)

The ability to understand a user's underlying needs is critical for conversational systems, especially with limited input from users in a conversation. Thus, in such a domain, Asking Clarification Questions (ACQs) to reveal users' true intent from their queries or utterances arise as an essential task. However, it is noticeable that a key limitation of the existing ACQs studies is their incomparability, from inconsistent use of data, distinct experimental setups and evaluation strategies. Therefore, in this paper, to assist the development of ACQs techniques, we comprehensively analyse the current ACQs research status, which offers a detailed comparison of publicly available datasets, and discusses the applied evaluation metrics, joined with benchmarks for multiple ACQs-related tasks. In particular, given a thorough analysis of the ACQs task, we discuss a number of corresponding research directions for the investigation of ACQs as well as the development of conversational systems.