mlsanityData to 7 Oct 2026

Asking Clarifying Questions in Open-Domain Information-Seeking Conversations

arXiv 1907.06554 ↗First posted 15 Jul 2019cs.CL cs.AI cs.IRcited by 136

Not on the map. Why?

Cited by 21 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Users often fail to formulate their complex information needs in a single query. As a consequence, they may need to scan multiple result pages or reformulate their queries, which may be a frustrating experience. Alternatively, systems can improve user satisfaction by proactively asking questions of the users to clarify their information needs. Asking clarifying questions is especially important in conversational systems since they can only return a limited number of (often only one) result(s). In this paper, we formulate the task of asking clarifying questions in open-domain information-seeking conversational systems. To this end, we propose an offline evaluation methodology for the task and collect a dataset, called Qulac, through crowdsourcing. Our dataset is built on top of the TREC Web Track 2009-2012 data and consists of over 10K question-answer pairs for 198 TREC topics with 762 facets. Our experiments on an oracle model demonstrate that asking only one good question leads to over 170% retrieval performance improvement in terms of P@1, which clearly demonstrates the potential impact of the task. We further propose a retrieval framework consisting of three components: question retrieval, question selection, and document retrieval. In particular, our question selection model takes into account the original query and previous question-answer interactions while selecting the next question. Our model significantly outperforms competitive baselines. To foster research in this area, we have made Qulac publicly available.

Key facts. Cited by 136 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Jul 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 20; 2022: 10; 2023: 13; 2024: 23; 2025: 24; 2026: 46.

136 papers.

Sort:

By area:

  1. 2021Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answeringcited by 178
  2. 2024Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Modelscited by 150
  3. 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 116
  4. 2023Let the LLMs Talk: Simulating Human-to-Human Conversational QA via Zero-Shot LLM-to-LLM Interactionscited by 70
  5. 2023A Survey on Proactive Dialogue Systems: Problems, Methods, and Prospectscited by 66
  6. 2021Building and Evaluating Open-Domain Dialogue Corpora with Clarifying Questionscited by 63
  7. 2024CRUD-RAG: A Comprehensive Chinese Benchmark for Retrieval-Augmented Generation of Large Language Modelscited by 50
  8. 2024Towards Human-centered Proactive Conversational Agentscited by 48
  9. 2023Knowledge-enhanced Mixed-initiative Dialogue System for Emotional Support Conversationscited by 41
  10. 2025Magentic-UI: Towards Human-in-the-loop Agentic Systemscited by 41
  11. 2021Ask what's missing and what's useful: Improving Clarification Question Generation using Global Knowledgecited by 40
  12. 2022ClarifyDelphi: Reinforced Clarification Questions with Defeasibility Rewards for Social and Moral Situationscited by 36
38 more
  1. 2023Active Preference Inference using Language Models and Probabilistic Reasoningcited by 29
  2. 2022INSCIT: Information-Seeking Conversations with Mixed-Initiative Interactionscited by 27
  3. 2022Task Ambiguity in Humans and Language Modelscited by 27
  4. 2021On the Calibration and Uncertainty of Neural Learning to Rank Modelscited by 24
  5. 2022Evaluating Mixed-initiative Conversational Search Systems via User Simulationcited by 24
  6. 2023Asking Clarification Questions to Handle Ambiguity in Open-Domain QAcited by 24
  7. 2023Probing the Multi-turn Planning Capabilities of LLMs via 20 Question Gamescited by 24
  8. 2021Asking It All: Generating Contextualized Questions for any Semantic Rolecited by 22
  9. 2021ISEEQ: Information Seeking Question Generation using Dynamic Meta-Information Retrieval and Knowledge Graphscited by 22
  10. 2024A Survey of Conversational Searchcited by 22
  11. 2025From Passive to Active Reasoning: Can Large Language Models Ask the Right Questions under Incomplete Information?cited by 21
  12. 2025DRAGged into Conflicts: Detecting and Addressing Conflicting Sources in Search-Augmented LLMscited by 21
  13. 2024STYLE: Improving Domain Transferability of Asking Clarification Questions in Large Language Model Powered Conversational Agentscited by 17
  14. 2022Python Code Generation by Asking Clarification Questionscited by 16
  15. 2023Interactive-Chain-Prompting: Ambiguity Resolution for Crosslingual Conditional Generation with Interactioncited by 16
  16. 2023Query Performance Prediction: From Ad-hoc to Conversational Searchcited by 16
  17. 2026HiL-Bench (Human-in-Loop Benchmark): Do Agents Know When to Ask for Help?cited by 16
  18. 2024RankMamba: Benchmarking Mamba's Document Ranking Performance in the Era of Transformerscited by 15
  19. 2026Ask or Assume? Uncertainty-Aware Clarification-Seeking in Coding Agentscited by 15
  20. 2021Dialogue History Matters! Personalized Response Selectionin Multi-turn Retrieval-based Chatbotscited by 14
  21. 2023Zero-shot Clarifying Question Generation for Conversational Searchcited by 14
  22. 2023Transforming Human-Centered AI Collaboration: Redefining Embodied Agents Capabilities through Interactive Grounded Language Instructionscited by 13
  23. 2026AgentIR: Reasoning-Aware Retrieval for Deep Research Agentscited by 13
  24. 2021Analysing Mixed Initiatives and Search Strategies during Conversational Searchcited by 12
  25. 2024Contextualized Evaluations: Judging Language Model Responses to Underspecified Queriescited by 12
  26. 2025InteractComp: Evaluating Search Agents With Ambiguous Queriescited by 12
  27. 2026When and What to Ask: AskBench and Rubric-Guided RLVR for LLM Clarificationcited by 12
  28. 2021Context-Aware Target Apps Selection and Recommendation for Enhancing Personal Mobile Assistantscited by 11
  29. 2021Wizard of Search Engine: Access to Information Through Conversations with Search Enginescited by 11
  30. 2023Instruction Clarification Requests in Multimodal Collaborative Dialogue Games: Tasks, and an Analysis of the CoDraw Datasetcited by 11
  31. 2024Empowering Language Models with Active Inquiry for Deeper Understandingcited by 11
  32. 2026Trajectory2Task: Training Robust Tool-Calling Agents with Synthesized Yet Verifiable Data for Complex User Intentscited by 11
  33. 2021Diverse and Specific Clarification Question Generation with Keywordscited by 10
  34. 2021Generating Usage-related Questions for Preference Elicitation in Conversational Recommender Systemscited by 10
  35. 2022Collecting Interactive Multi-modal Datasets for Grounded Language Understandingcited by 10
  36. 2025Interpersonal Memory Matters: A New Task for Proactive Dialogue Utilizing Conversational Historycited by 10
  37. 2025HumanAgencyBench: Scalable Evaluation of Human Agency Support in AI Assistantscited by 10
  38. 2021Database Search Results Disambiguation for Task-Oriented Dialog Systemscited by 9

50 most cited of 136 All 136 (JSON)