# Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Information

arXiv: https://arxiv.org/abs/1805.04655 · HTML: https://mlsanity.com/paper/1805.04655/

## Key facts

- Cited by 115 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 May 2018.

## Built on by (most cited first, 30 of 115)

- [The Art of Saying No: Contextual Noncompliance in Language Models](https://mlsanity.com/paper/2407.12043/) (2024)
- [STaR-GATE: Teaching Language Models to Ask Clarifying Questions](https://mlsanity.com/paper/2403.19154/) (2024)
- [Clarify When Necessary: Resolving Ambiguity Through Interaction with LMs](https://mlsanity.com/paper/2311.09469/) (2023)
- [What Evidence Do Language Models Find Convincing?](https://mlsanity.com/paper/2402.11782/) (2024)
- [Generating Literal and Implied Subquestions to Fact-check Complex Claims](https://mlsanity.com/paper/2205.06938/) (2022)
- [Building and Evaluating Open-Domain Dialogue Corpora with Clarifying Questions](https://mlsanity.com/paper/2109.05794/) (2021)
- [Ask-before-Plan: Proactive Language Agents for Real-World Planning](https://mlsanity.com/paper/2406.12639/) (2024)
- [Grounding 'Grounding' in NLP](https://mlsanity.com/paper/2106.02192/) (2021)
- [Automatic Generation of Socratic Subquestions for Teaching Math Word Problems](https://mlsanity.com/paper/2211.12835/) (2022)
- [Neural Approaches to Conversational Information Retrieval](https://mlsanity.com/paper/2201.05176/) (2022)
- [Learning to Ask: When LLM Agents Meet Unclear Instruction](https://mlsanity.com/paper/2409.00557/) (2024)
- [Structured Uncertainty guided Clarification for LLM Agents](https://mlsanity.com/paper/2511.08798/) (2025)
- [Putting Humans in the Natural Language Processing Loop: A Survey](https://mlsanity.com/paper/2103.04044/) (2021)
- [Ask what's missing and what's useful: Improving Clarification Question Generation using Global Knowledge](https://mlsanity.com/paper/2104.06828/) (2021)
- [Controllable Open-ended Question Generation with A New Question Type Ontology](https://mlsanity.com/paper/2107.00152/) (2021)
- [ClarifyDelphi: Reinforced Clarification Questions with Defeasibility Rewards for Social and Moral Situations](https://mlsanity.com/paper/2212.10409/) (2022)
- [Active Preference Inference using Language Models and Probabilistic Reasoning](https://mlsanity.com/paper/2312.12009/) (2023)
- [HiL-Bench (Human-in-Loop Benchmark): Do Agents Know When to Ask for Help?](https://mlsanity.com/paper/2604.09408/) (2026)
- [Asking Clarification Questions to Handle Ambiguity in Open-Domain QA](https://mlsanity.com/paper/2305.13808/) (2023)
- [Evaluating Mixed-initiative Conversational Search Systems via User Simulation](https://mlsanity.com/paper/2204.08046/) (2022)
- [On Overcoming Miscalibrated Conversational Priors in LLM-based Chatbots](https://mlsanity.com/paper/2406.01633/) (2024)
- [Interactive Grounded Language Understanding in a Collaborative Environment: IGLU 2021](https://mlsanity.com/paper/2205.02388/) (2022)
- [ISEEQ: Information Seeking Question Generation using Dynamic Meta-Information Retrieval and Knowledge Graphs](https://mlsanity.com/paper/2112.07622/) (2021)
- [A Survey of Conversational Search](https://mlsanity.com/paper/2410.15576/) (2024)
- [Open-domain clarification question generation without question examples](https://mlsanity.com/paper/2110.09779/) (2021)
- [Post-Abstention: Towards Reliably Re-Attempting the Abstained Instances in QA](https://mlsanity.com/paper/2305.01812/) (2023)
- [INTERS: Unlocking the Power of Large Language Models in Search with Instruction Tuning](https://mlsanity.com/paper/2401.06532/) (2024)
- [CLARINET: Augmenting Language Models to Ask Clarification Questions for Retrieval](https://mlsanity.com/paper/2405.15784/) (2024)
- [Ask or Assume? Uncertainty-Aware Clarification-Seeking in Coding Agents](https://mlsanity.com/paper/2603.26233/) (2026)
- [Python Code Generation by Asking Clarification Questions](https://mlsanity.com/paper/2212.09885/) (2022)

## Abstract (arXiv)

Inquiry is fundamental to communication, and machines cannot effectively collaborate with humans unless they can ask questions. In this work, we build a neural network model for the task of ranking clarification questions. Our model is inspired by the idea of expected value of perfect information: a good question is one whose expected answer will be useful. We study this problem using data from StackExchange, a plentiful online resource in which people routinely ask clarifying questions to posts so that they can better offer assistance to the original poster. We create a dataset of clarification questions consisting of ~77K posts paired with a clarification question (and answer) from three domains of StackExchange: askubuntu, unix and superuser. We evaluate our model on 500 samples of this dataset against expert human judgments and demonstrate significant improvements over controlled baselines.