mlsanityData to 7 Oct 2026

Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Information

arXiv 1805.04655 ↗First posted 12 May 2018cs.CLcited by 110

Not on the map. Why?

Abstract (arXiv)

Inquiry is fundamental to communication, and machines cannot effectively collaborate with humans unless they can ask questions. In this work, we build a neural network model for the task of ranking clarification questions. Our model is inspired by the idea of expected value of perfect information: a good question is one whose expected answer will be useful. We study this problem using data from StackExchange, a plentiful online resource in which people routinely ask clarifying questions to posts so that they can better offer assistance to the original poster. We create a dataset of clarification questions consisting of ~77K posts paired with a clarification question (and answer) from three domains of StackExchange: askubuntu, unix and superuser. We evaluate our model on 500 samples of this dataset against expert human judgments and demonstrate significant improvements over controlled baselines.

Key facts. Cited by 110 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 16; 2022: 13; 2023: 15; 2024: 27; 2025: 11; 2026: 28.

110 papers.

Sort:

By area:

  1. 2024The Art of Saying No: Contextual Noncompliance in Language Modelscited by 100
  2. 2024STaR-GATE: Teaching Language Models to Ask Clarifying Questionscited by 99
  3. 2023Clarify When Necessary: Resolving Ambiguity Through Interaction with LMscited by 93
  4. 2024What Evidence Do Language Models Find Convincing?cited by 74
  5. 2022Generating Literal and Implied Subquestions to Fact-check Complex Claimscited by 66
  6. 2021Building and Evaluating Open-Domain Dialogue Corpora with Clarifying Questionscited by 63
  7. 2024Ask-before-Plan: Proactive Language Agents for Real-World Planningcited by 59
  8. 2021Grounding 'Grounding' in NLPcited by 44
  9. 2022Automatic Generation of Socratic Subquestions for Teaching Math Word Problemscited by 42
  10. 2022Neural Approaches to Conversational Information Retrievalcited by 41
  11. 2021Putting Humans in the Natural Language Processing Loop: A Surveycited by 40
  12. 2021Ask what's missing and what's useful: Improving Clarification Question Generation using Global Knowledgecited by 40
38 more
  1. 2024Learning to Ask: When LLM Agents Meet Unclear Instructioncited by 38
  2. 2021Controllable Open-ended Question Generation with A New Question Type Ontologycited by 36
  3. 2022ClarifyDelphi: Reinforced Clarification Questions with Defeasibility Rewards for Social and Moral Situationscited by 36
  4. 2025Structured Uncertainty guided Clarification for LLM Agentscited by 34
  5. 2023Active Preference Inference using Language Models and Probabilistic Reasoningcited by 29
  6. 2022Evaluating Mixed-initiative Conversational Search Systems via User Simulationcited by 24
  7. 2023Asking Clarification Questions to Handle Ambiguity in Open-Domain QAcited by 24
  8. 2022Interactive Grounded Language Understanding in a Collaborative Environment: IGLU 2021cited by 23
  9. 2024On Overcoming Miscalibrated Conversational Priors in LLM-based Chatbotscited by 23
  10. 2021ISEEQ: Information Seeking Question Generation using Dynamic Meta-Information Retrieval and Knowledge Graphscited by 22
  11. 2024A Survey of Conversational Searchcited by 22
  12. 2021Open-domain clarification question generation without question examplescited by 20
  13. 2023Post-Abstention: Towards Reliably Re-Attempting the Abstained Instances in QAcited by 20
  14. 2024INTERS: Unlocking the Power of Large Language Models in Search with Instruction Tuningcited by 19
  15. 2024CLARINET: Augmenting Language Models to Ask Clarification Questions for Retrievalcited by 18
  16. 2022Python Code Generation by Asking Clarification Questionscited by 16
  17. 2026Calibrate-Then-Act: Cost-Aware Exploration in LLM Agentscited by 16
  18. 2026HiL-Bench (Human-in-Loop Benchmark): Do Agents Know When to Ask for Help?cited by 16
  19. 2021NeurIPS 2021 Competition IGLU: Interactive Grounded Language Understanding in a Collaborative Environmentcited by 15
  20. 2026Ask or Assume? Uncertainty-Aware Clarification-Seeking in Coding Agentscited by 15
  21. 2022IGLU 2022: Interactive Grounded Language Understanding in a Collaborative Environment at NeurIPS 2022cited by 14
  22. 2023Zero-shot Clarifying Question Generation for Conversational Searchcited by 14
  23. 2023Context Aware Query Rewriting for Text Rankers using LLMcited by 14
  24. 2024InfoLossQA: Characterizing and Recovering Information Loss in Text Simplificationcited by 14
  25. 2021A recipe for annotating grounded clarificationscited by 13
  26. 2021Analysing Mixed Initiatives and Search Strategies during Conversational Searchcited by 12
  27. 2021A Framework for Learning to Request Rich and Contextually Useful Information from Humanscited by 12
  28. 2022Quiz Design Task: Helping Teachers Create Quizzes with Automated Question Generationcited by 12
  29. 2023FOLLOWUPQG: Towards Information-Seeking Follow-up Question Generationcited by 12
  30. 2024Contextualized Evaluations: Judging Language Model Responses to Underspecified Queriescited by 12
  31. 2022Discord Questions: A Computational Approach To Diversity Analysis in News Coveragecited by 11
  32. 2023Instruction Clarification Requests in Multimodal Collaborative Dialogue Games: Tasks, and an Analysis of the CoDraw Datasetcited by 11
  33. 2021Diverse and Specific Clarification Question Generation with Keywordscited by 10
  34. 2021Generating Usage-related Questions for Preference Elicitation in Conversational Recommender Systemscited by 10
  35. 2022Collecting Interactive Multi-modal Datasets for Grounded Language Understandingcited by 10
  36. 2025HumanAgencyBench: Scalable Evaluation of Human Agency Support in AI Assistantscited by 10
  37. 2021Database Search Results Disambiguation for Task-Oriented Dialog Systemscited by 9
  38. 2023Towards a Benchmark for Scientific Understanding in Humans and Machinescited by 9

50 most cited of 110 All 110 (JSON)