# CLAIM: Leading Open-domain Active Clarification of Large Language Models with Uncertainty Measurement

arXiv: https://arxiv.org/abs/2608.11631 · HTML: https://mlsanity.com/paper/2608.11631/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 40 references that are in the map.
- First posted on arXiv 12 Aug 2026.

## Builds on (most cited first, 16 of 16)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation](https://mlsanity.com/paper/2302.09664/) (2023)
- [Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMs](https://mlsanity.com/paper/2406.15927/) (2024)
- [Modeling Future Conversation Turns to Teach LLMs to Ask Clarifying Questions](https://mlsanity.com/paper/2410.13788/) (2024)
- [QuestBench: Can LLMs ask the right question to acquire information in reasoning tasks?](https://mlsanity.com/paper/2503.22674/) (2025)
- [ConvAI3: Generating Clarifying Questions for Open-Domain Dialogue Systems (ClariQ)](https://mlsanity.com/paper/2009.11352/) (2020)
- [ClarQ-LLM: A Benchmark for Models Clarifying and Requesting Information in Task-Oriented Dialog](https://mlsanity.com/paper/2409.06097/) (2024)
- [From Passive to Active Reasoning: Can Large Language Models Ask the Right Questions under Incomplete Information?](https://mlsanity.com/paper/2506.08295/) (2025)
- [CLARINET: Augmenting Language Models to Ask Clarification Questions for Retrieval](https://mlsanity.com/paper/2405.15784/) (2024)
- [Pre-Trained Policy Discriminators are General Reward Models](https://mlsanity.com/paper/2507.05197/) (2025)
- [Question Asking as Program Generation](https://mlsanity.com/paper/1711.06351/) (2017)
- [Asking Clarifying Questions for Preference Elicitation With Large Language Models](https://mlsanity.com/paper/2510.12015/) (2025)
- [Large Language Models Should Ask Clarifying Questions to Increase Confidence in Generated Code](https://mlsanity.com/paper/2308.13507/) (2023)

## Abstract (arXiv)

In open-domain human-computer interaction scenarios, large language models (LLMs) frequently encounter user queries that are ambiguous or incomplete. In such cases, directly producing an answer often leads to overgeneralized, erroneous, or low-information responses. In contrast, asking clarifying questions can substantially improve interaction quality. However, existing approaches still rely heavily on manually annotated data or preference alignment to address two fundamental challenges: when clarification is necessary, and which aspect of the query should be clarified. This reliance incurs high annotation costs and limits generalization. To address these challenges, we propose CLAIM, an uncertainty-driven framework for active clarification learning in open-domain settings. CLAIM eliminates the need for explicit human preference annotations by quantifying query uncertainty through the entropy induced by answer disagreements across multiple models. This uncertainty signal is then used to construct high-quality synthetic data, enabling the training of a unified clarification decision model through a combination of supervised learning and reinforcement learning. Specifically, we propose an entropy-driven synthetic data generation pipeline that integrates entropy-based uncertainty estimation with semantic clustering and reasoning-based judgments, enabling reliable automatic annotation of clarification requirements. To train CLAIM, we formulate the clarification process as a structured decision generation problem and adopt a training paradigm that combines supervised fine-tuning (SFT) with group-relative policy optimization (GRPO). Experimental results demonstrate that CLAIM can learn stable and generalizable clarification strategies without relying on manually labeled data, offering a low-cost and robust solution for proactive understanding in real-world open-domain interactions with LLMs.